2023-07-24 07:53:12,641 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-07-24 07:53:12,901 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97 2023-07-24 07:53:12,916 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-24 07:53:12,916 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f, deleteOnExit=true 2023-07-24 07:53:12,916 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-24 07:53:12,917 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/test.cache.data in system properties and HBase conf 2023-07-24 07:53:12,917 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/hadoop.tmp.dir in system properties and HBase conf 2023-07-24 07:53:12,918 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/hadoop.log.dir in system properties and HBase conf 2023-07-24 07:53:12,918 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-24 07:53:12,919 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-24 07:53:12,919 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-24 07:53:13,040 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-24 07:53:13,509 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-24 07:53:13,515 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-24 07:53:13,515 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-24 07:53:13,516 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-24 07:53:13,516 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-24 07:53:13,517 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-24 07:53:13,517 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-24 07:53:13,518 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-24 07:53:13,518 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-24 07:53:13,519 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-24 07:53:13,519 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/nfs.dump.dir in system properties and HBase conf 2023-07-24 07:53:13,520 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/java.io.tmpdir in system properties and HBase conf 2023-07-24 07:53:13,520 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-24 07:53:13,520 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-24 07:53:13,521 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-24 07:53:14,169 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-24 07:53:14,174 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-24 07:53:14,518 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-24 07:53:14,697 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-24 07:53:14,715 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 07:53:14,751 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 07:53:14,784 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/java.io.tmpdir/Jetty_localhost_43049_hdfs____.hz5dvk/webapp 2023-07-24 07:53:14,926 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43049 2023-07-24 07:53:14,944 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-24 07:53:14,944 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-24 07:53:15,492 WARN [Listener at localhost/34211] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 07:53:15,564 WARN [Listener at localhost/34211] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-24 07:53:15,588 WARN [Listener at localhost/34211] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 07:53:15,596 INFO [Listener at localhost/34211] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 07:53:15,602 INFO [Listener at localhost/34211] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/java.io.tmpdir/Jetty_localhost_39963_datanode____3ejipj/webapp 2023-07-24 07:53:15,709 INFO [Listener at localhost/34211] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39963 2023-07-24 07:53:16,059 WARN [Listener at localhost/41725] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 07:53:16,097 WARN [Listener at localhost/41725] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-24 07:53:16,113 WARN [Listener at localhost/41725] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 07:53:16,115 INFO [Listener at localhost/41725] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 07:53:16,129 INFO [Listener at localhost/41725] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/java.io.tmpdir/Jetty_localhost_34143_datanode____4y3fti/webapp 2023-07-24 07:53:16,234 INFO [Listener at localhost/41725] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34143 2023-07-24 07:53:16,250 WARN [Listener at localhost/36759] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 07:53:16,287 WARN [Listener at localhost/36759] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-24 07:53:16,293 WARN [Listener at localhost/36759] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 07:53:16,296 INFO [Listener at localhost/36759] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 07:53:16,310 INFO [Listener at localhost/36759] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/java.io.tmpdir/Jetty_localhost_46205_datanode____.bx18ys/webapp 2023-07-24 07:53:16,424 INFO [Listener at localhost/36759] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46205 2023-07-24 07:53:16,439 WARN [Listener at localhost/42537] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 07:53:16,663 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x26f8b31a1ed20381: Processing first storage report for DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa from datanode cda8ebfd-f055-417e-8edc-61fd19134adf 2023-07-24 07:53:16,665 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x26f8b31a1ed20381: from storage DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa node DatanodeRegistration(127.0.0.1:33477, datanodeUuid=cda8ebfd-f055-417e-8edc-61fd19134adf, infoPort=33091, infoSecurePort=0, ipcPort=41725, storageInfo=lv=-57;cid=testClusterID;nsid=1457520341;c=1690185194253), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-24 07:53:16,665 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x62ffe2b894f14d2f: Processing first storage report for DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7 from datanode b4241ba3-003c-4dbd-8dcc-7026b912408a 2023-07-24 07:53:16,665 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x62ffe2b894f14d2f: from storage DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7 node DatanodeRegistration(127.0.0.1:44119, datanodeUuid=b4241ba3-003c-4dbd-8dcc-7026b912408a, infoPort=35331, infoSecurePort=0, ipcPort=42537, storageInfo=lv=-57;cid=testClusterID;nsid=1457520341;c=1690185194253), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 07:53:16,666 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd90b185b9a749404: Processing first storage report for DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed from datanode 1d2b2de7-56a0-45bb-9a79-d5935e59fefd 2023-07-24 07:53:16,666 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd90b185b9a749404: from storage DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed node DatanodeRegistration(127.0.0.1:40639, datanodeUuid=1d2b2de7-56a0-45bb-9a79-d5935e59fefd, infoPort=36003, infoSecurePort=0, ipcPort=36759, storageInfo=lv=-57;cid=testClusterID;nsid=1457520341;c=1690185194253), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 07:53:16,666 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x26f8b31a1ed20381: Processing first storage report for DS-185f119d-5edb-419f-8144-22f6880e8194 from datanode cda8ebfd-f055-417e-8edc-61fd19134adf 2023-07-24 07:53:16,666 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x26f8b31a1ed20381: from storage DS-185f119d-5edb-419f-8144-22f6880e8194 node DatanodeRegistration(127.0.0.1:33477, datanodeUuid=cda8ebfd-f055-417e-8edc-61fd19134adf, infoPort=33091, infoSecurePort=0, ipcPort=41725, storageInfo=lv=-57;cid=testClusterID;nsid=1457520341;c=1690185194253), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 07:53:16,666 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x62ffe2b894f14d2f: Processing first storage report for DS-e8aa4895-0942-480a-916a-87092814c14f from datanode b4241ba3-003c-4dbd-8dcc-7026b912408a 2023-07-24 07:53:16,667 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x62ffe2b894f14d2f: from storage DS-e8aa4895-0942-480a-916a-87092814c14f node DatanodeRegistration(127.0.0.1:44119, datanodeUuid=b4241ba3-003c-4dbd-8dcc-7026b912408a, infoPort=35331, infoSecurePort=0, ipcPort=42537, storageInfo=lv=-57;cid=testClusterID;nsid=1457520341;c=1690185194253), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-24 07:53:16,667 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd90b185b9a749404: Processing first storage report for DS-48aeaa99-eb4c-42f3-b317-644b2d280cc2 from datanode 1d2b2de7-56a0-45bb-9a79-d5935e59fefd 2023-07-24 07:53:16,667 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd90b185b9a749404: from storage DS-48aeaa99-eb4c-42f3-b317-644b2d280cc2 node DatanodeRegistration(127.0.0.1:40639, datanodeUuid=1d2b2de7-56a0-45bb-9a79-d5935e59fefd, infoPort=36003, infoSecurePort=0, ipcPort=36759, storageInfo=lv=-57;cid=testClusterID;nsid=1457520341;c=1690185194253), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 07:53:16,903 DEBUG [Listener at localhost/42537] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97 2023-07-24 07:53:16,970 INFO [Listener at localhost/42537] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/zookeeper_0, clientPort=55594, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-24 07:53:16,990 INFO [Listener at localhost/42537] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55594 2023-07-24 07:53:17,001 INFO [Listener at localhost/42537] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 07:53:17,005 INFO [Listener at localhost/42537] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 07:53:17,696 INFO [Listener at localhost/42537] util.FSUtils(471): Created version file at hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99 with version=8 2023-07-24 07:53:17,696 INFO [Listener at localhost/42537] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/hbase-staging 2023-07-24 07:53:18,050 INFO [Listener at localhost/42537] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-24 07:53:18,496 INFO [Listener at localhost/42537] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 07:53:18,528 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:18,529 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:18,529 INFO [Listener at localhost/42537] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 07:53:18,529 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:18,529 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 07:53:18,674 INFO [Listener at localhost/42537] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 07:53:18,754 DEBUG [Listener at localhost/42537] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-24 07:53:18,849 INFO [Listener at localhost/42537] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37123 2023-07-24 07:53:18,860 INFO [Listener at localhost/42537] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 07:53:18,861 INFO [Listener at localhost/42537] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 07:53:18,882 INFO [Listener at localhost/42537] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37123 connecting to ZooKeeper ensemble=127.0.0.1:55594 2023-07-24 07:53:18,924 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:371230x0, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 07:53:18,928 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:37123-0x1019651e89e0000 connected 2023-07-24 07:53:18,955 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 07:53:18,956 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 07:53:18,960 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 07:53:18,968 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37123 2023-07-24 07:53:18,968 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37123 2023-07-24 07:53:18,969 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37123 2023-07-24 07:53:18,969 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37123 2023-07-24 07:53:18,969 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37123 2023-07-24 07:53:18,975 INFO [Listener at localhost/42537] master.HMaster(444): hbase.rootdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99, hbase.cluster.distributed=false 2023-07-24 07:53:19,046 INFO [Listener at localhost/42537] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 07:53:19,046 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:19,047 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:19,047 INFO [Listener at localhost/42537] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 07:53:19,047 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:19,047 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 07:53:19,052 INFO [Listener at localhost/42537] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 07:53:19,055 INFO [Listener at localhost/42537] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46877 2023-07-24 07:53:19,057 INFO [Listener at localhost/42537] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 07:53:19,063 DEBUG [Listener at localhost/42537] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 07:53:19,064 INFO [Listener at localhost/42537] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 07:53:19,066 INFO [Listener at localhost/42537] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 07:53:19,068 INFO [Listener at localhost/42537] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46877 connecting to ZooKeeper ensemble=127.0.0.1:55594 2023-07-24 07:53:19,071 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:468770x0, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 07:53:19,072 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): regionserver:468770x0, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 07:53:19,074 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46877-0x1019651e89e0001 connected 2023-07-24 07:53:19,074 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 07:53:19,075 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 07:53:19,076 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46877 2023-07-24 07:53:19,076 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46877 2023-07-24 07:53:19,077 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46877 2023-07-24 07:53:19,078 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46877 2023-07-24 07:53:19,079 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46877 2023-07-24 07:53:19,092 INFO [Listener at localhost/42537] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 07:53:19,092 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:19,092 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:19,093 INFO [Listener at localhost/42537] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 07:53:19,093 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:19,094 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 07:53:19,094 INFO [Listener at localhost/42537] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 07:53:19,099 INFO [Listener at localhost/42537] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46543 2023-07-24 07:53:19,099 INFO [Listener at localhost/42537] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 07:53:19,103 DEBUG [Listener at localhost/42537] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 07:53:19,104 INFO [Listener at localhost/42537] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 07:53:19,106 INFO [Listener at localhost/42537] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 07:53:19,109 INFO [Listener at localhost/42537] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46543 connecting to ZooKeeper ensemble=127.0.0.1:55594 2023-07-24 07:53:19,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:465430x0, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 07:53:19,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46543-0x1019651e89e0002 connected 2023-07-24 07:53:19,115 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 07:53:19,116 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 07:53:19,117 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 07:53:19,118 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46543 2023-07-24 07:53:19,118 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46543 2023-07-24 07:53:19,119 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46543 2023-07-24 07:53:19,122 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46543 2023-07-24 07:53:19,124 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46543 2023-07-24 07:53:19,137 INFO [Listener at localhost/42537] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 07:53:19,137 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:19,137 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:19,137 INFO [Listener at localhost/42537] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 07:53:19,137 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 07:53:19,137 INFO [Listener at localhost/42537] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 07:53:19,137 INFO [Listener at localhost/42537] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 07:53:19,139 INFO [Listener at localhost/42537] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37809 2023-07-24 07:53:19,140 INFO [Listener at localhost/42537] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 07:53:19,142 DEBUG [Listener at localhost/42537] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 07:53:19,143 INFO [Listener at localhost/42537] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 07:53:19,145 INFO [Listener at localhost/42537] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 07:53:19,147 INFO [Listener at localhost/42537] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37809 connecting to ZooKeeper ensemble=127.0.0.1:55594 2023-07-24 07:53:19,151 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:378090x0, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 07:53:19,152 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37809-0x1019651e89e0003 connected 2023-07-24 07:53:19,152 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 07:53:19,153 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 07:53:19,154 DEBUG [Listener at localhost/42537] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 07:53:19,154 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37809 2023-07-24 07:53:19,155 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37809 2023-07-24 07:53:19,158 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37809 2023-07-24 07:53:19,159 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37809 2023-07-24 07:53:19,160 DEBUG [Listener at localhost/42537] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37809 2023-07-24 07:53:19,161 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37123,1690185197841 2023-07-24 07:53:19,171 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-24 07:53:19,172 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37123,1690185197841 2023-07-24 07:53:19,191 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 07:53:19,191 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 07:53:19,191 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 07:53:19,191 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 07:53:19,193 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 07:53:19,194 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 07:53:19,195 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37123,1690185197841 from backup master directory 2023-07-24 07:53:19,196 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 07:53:19,201 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37123,1690185197841 2023-07-24 07:53:19,201 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-24 07:53:19,202 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 07:53:19,202 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,37123,1690185197841 2023-07-24 07:53:19,205 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-24 07:53:19,207 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-24 07:53:19,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/hbase.id with ID: 51fd3aa7-8856-434c-8276-78c3430cd418 2023-07-24 07:53:19,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 07:53:19,364 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 07:53:19,461 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4741057f to 127.0.0.1:55594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 07:53:19,494 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65761e74, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-24 07:53:19,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 07:53:19,523 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-24 07:53:19,541 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-24 07:53:19,541 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-24 07:53:19,544 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-24 07:53:19,549 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-24 07:53:19,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 07:53:19,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/data/master/store-tmp 2023-07-24 07:53:19,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:19,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-24 07:53:19,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 07:53:19,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 07:53:19,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-24 07:53:19,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 07:53:19,630 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 07:53:19,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 07:53:19,632 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/WALs/jenkins-hbase4.apache.org,37123,1690185197841 2023-07-24 07:53:19,653 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37123%2C1690185197841, suffix=, logDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/WALs/jenkins-hbase4.apache.org,37123,1690185197841, archiveDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/oldWALs, maxLogs=10 2023-07-24 07:53:19,711 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44119,DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7,DISK] 2023-07-24 07:53:19,711 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40639,DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed,DISK] 2023-07-24 07:53:19,711 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33477,DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa,DISK] 2023-07-24 07:53:19,719 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 07:53:19,788 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/WALs/jenkins-hbase4.apache.org,37123,1690185197841/jenkins-hbase4.apache.org%2C37123%2C1690185197841.1690185199664 2023-07-24 07:53:19,788 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40639,DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed,DISK], DatanodeInfoWithStorage[127.0.0.1:33477,DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa,DISK], DatanodeInfoWithStorage[127.0.0.1:44119,DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7,DISK]] 2023-07-24 07:53:19,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:53:19,789 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:19,792 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 07:53:19,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 07:53:19,850 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-24 07:53:19,857 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-24 07:53:19,881 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-24 07:53:19,896 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:19,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-24 07:53:19,905 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-24 07:53:19,919 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 07:53:19,923 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:53:19,924 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66273071, jitterRate=-0.012454286217689514}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 07:53:19,924 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 07:53:19,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-24 07:53:19,946 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:19,946 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:19,948 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:19,950 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-24 07:53:19,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 33 msec 2023-07-24 07:53:19,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-24 07:53:20,012 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-24 07:53:20,018 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-24 07:53:20,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-24 07:53:20,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-24 07:53:20,049 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-07-24 07:53:20,055 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-24 07:53:20,059 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-24 07:53:20,062 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 07:53:20,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-24 07:53:20,064 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-24 07:53:20,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-24 07:53:20,082 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 07:53:20,082 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 07:53:20,082 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 07:53:20,082 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 07:53:20,083 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 07:53:20,083 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,37123,1690185197841, sessionid=0x1019651e89e0000, setting cluster-up flag (Was=false) 2023-07-24 07:53:20,099 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 07:53:20,105 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-24 07:53:20,106 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37123,1690185197841 2023-07-24 07:53:20,112 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 07:53:20,118 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-24 07:53:20,119 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37123,1690185197841 2023-07-24 07:53:20,122 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.hbase-snapshot/.tmp 2023-07-24 07:53:20,165 INFO [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(951): ClusterId : 51fd3aa7-8856-434c-8276-78c3430cd418 2023-07-24 07:53:20,165 INFO [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(951): ClusterId : 51fd3aa7-8856-434c-8276-78c3430cd418 2023-07-24 07:53:20,165 INFO [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(951): ClusterId : 51fd3aa7-8856-434c-8276-78c3430cd418 2023-07-24 07:53:20,172 DEBUG [RS:2;jenkins-hbase4:37809] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 07:53:20,172 DEBUG [RS:1;jenkins-hbase4:46543] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 07:53:20,172 DEBUG [RS:0;jenkins-hbase4:46877] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 07:53:20,179 DEBUG [RS:2;jenkins-hbase4:37809] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 07:53:20,179 DEBUG [RS:1;jenkins-hbase4:46543] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 07:53:20,179 DEBUG [RS:0;jenkins-hbase4:46877] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 07:53:20,179 DEBUG [RS:1;jenkins-hbase4:46543] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 07:53:20,179 DEBUG [RS:2;jenkins-hbase4:37809] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 07:53:20,179 DEBUG [RS:0;jenkins-hbase4:46877] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 07:53:20,183 DEBUG [RS:2;jenkins-hbase4:37809] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 07:53:20,183 DEBUG [RS:0;jenkins-hbase4:46877] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 07:53:20,183 DEBUG [RS:1;jenkins-hbase4:46543] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 07:53:20,187 DEBUG [RS:0;jenkins-hbase4:46877] zookeeper.ReadOnlyZKClient(139): Connect 0x1bc1e74e to 127.0.0.1:55594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 07:53:20,187 DEBUG [RS:1;jenkins-hbase4:46543] zookeeper.ReadOnlyZKClient(139): Connect 0x0bb93727 to 127.0.0.1:55594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 07:53:20,187 DEBUG [RS:2;jenkins-hbase4:37809] zookeeper.ReadOnlyZKClient(139): Connect 0x5c963d3d to 127.0.0.1:55594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 07:53:20,199 DEBUG [RS:0;jenkins-hbase4:46877] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@15061a46, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-24 07:53:20,200 DEBUG [RS:0;jenkins-hbase4:46877] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11398c3e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 07:53:20,200 DEBUG [RS:2;jenkins-hbase4:37809] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5251c8df, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-24 07:53:20,201 DEBUG [RS:2;jenkins-hbase4:37809] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@566e564d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 07:53:20,201 DEBUG [RS:1;jenkins-hbase4:46543] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@57ec820, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-24 07:53:20,201 DEBUG [RS:1;jenkins-hbase4:46543] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1415c57, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 07:53:20,234 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:37809 2023-07-24 07:53:20,234 DEBUG [RS:1;jenkins-hbase4:46543] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:46543 2023-07-24 07:53:20,235 DEBUG [RS:0;jenkins-hbase4:46877] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46877 2023-07-24 07:53:20,239 INFO [RS:1;jenkins-hbase4:46543] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 07:53:20,239 INFO [RS:0;jenkins-hbase4:46877] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 07:53:20,240 INFO [RS:0;jenkins-hbase4:46877] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 07:53:20,239 INFO [RS:2;jenkins-hbase4:37809] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 07:53:20,241 DEBUG [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 07:53:20,240 INFO [RS:1;jenkins-hbase4:46543] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 07:53:20,241 INFO [RS:2;jenkins-hbase4:37809] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 07:53:20,241 DEBUG [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 07:53:20,241 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 07:53:20,243 INFO [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37123,1690185197841 with isa=jenkins-hbase4.apache.org/172.31.14.131:46877, startcode=1690185199045 2023-07-24 07:53:20,243 INFO [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37123,1690185197841 with isa=jenkins-hbase4.apache.org/172.31.14.131:37809, startcode=1690185199136 2023-07-24 07:53:20,243 INFO [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37123,1690185197841 with isa=jenkins-hbase4.apache.org/172.31.14.131:46543, startcode=1690185199091 2023-07-24 07:53:20,264 DEBUG [RS:1;jenkins-hbase4:46543] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 07:53:20,264 DEBUG [RS:0;jenkins-hbase4:46877] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 07:53:20,264 DEBUG [RS:2;jenkins-hbase4:37809] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 07:53:20,285 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:20,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 07:53:20,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 07:53:20,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 07:53:20,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 07:53:20,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-24 07:53:20,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 07:53:20,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,304 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185230304 2023-07-24 07:53:20,306 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-24 07:53:20,312 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:20,313 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-24 07:53:20,319 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-24 07:53:20,321 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-24 07:53:20,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-24 07:53:20,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-24 07:53:20,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-24 07:53:20,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-24 07:53:20,333 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59993, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 07:53:20,333 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36989, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 07:53:20,333 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53707, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 07:53:20,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,340 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-24 07:53:20,342 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-24 07:53:20,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-24 07:53:20,344 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37123] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 07:53:20,346 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-24 07:53:20,347 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-24 07:53:20,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690185200350,5,FailOnTimeoutGroup] 2023-07-24 07:53:20,352 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690185200352,5,FailOnTimeoutGroup] 2023-07-24 07:53:20,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-24 07:53:20,354 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37123] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 07:53:20,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,355 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,355 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37123] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 07:53:20,383 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-24 07:53:20,385 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-24 07:53:20,385 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99 2023-07-24 07:53:20,386 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(2830): Master is not running yet 2023-07-24 07:53:20,386 DEBUG [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(2830): Master is not running yet 2023-07-24 07:53:20,386 DEBUG [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(2830): Master is not running yet 2023-07-24 07:53:20,387 WARN [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-24 07:53:20,387 WARN [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-24 07:53:20,387 WARN [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-24 07:53:20,411 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:20,413 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-24 07:53:20,416 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/info 2023-07-24 07:53:20,417 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-24 07:53:20,417 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:20,418 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-24 07:53:20,420 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/rep_barrier 2023-07-24 07:53:20,421 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-24 07:53:20,422 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:20,422 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-24 07:53:20,424 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/table 2023-07-24 07:53:20,424 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-24 07:53:20,425 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:20,427 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740 2023-07-24 07:53:20,428 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740 2023-07-24 07:53:20,432 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-24 07:53:20,434 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-24 07:53:20,440 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:53:20,441 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61510782, jitterRate=-0.08341792225837708}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-24 07:53:20,442 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-24 07:53:20,442 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-24 07:53:20,442 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-24 07:53:20,442 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-24 07:53:20,442 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-24 07:53:20,442 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-24 07:53:20,444 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-24 07:53:20,444 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-24 07:53:20,451 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:20,451 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-24 07:53:20,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-24 07:53:20,478 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-24 07:53:20,480 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-24 07:53:20,488 INFO [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37123,1690185197841 with isa=jenkins-hbase4.apache.org/172.31.14.131:37809, startcode=1690185199136 2023-07-24 07:53:20,488 INFO [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37123,1690185197841 with isa=jenkins-hbase4.apache.org/172.31.14.131:46877, startcode=1690185199045 2023-07-24 07:53:20,488 INFO [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37123,1690185197841 with isa=jenkins-hbase4.apache.org/172.31.14.131:46543, startcode=1690185199091 2023-07-24 07:53:20,494 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37123] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:20,500 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37123] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:20,501 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99 2023-07-24 07:53:20,501 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37123] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:20,501 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:34211 2023-07-24 07:53:20,501 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-24 07:53:20,501 DEBUG [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99 2023-07-24 07:53:20,502 DEBUG [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:34211 2023-07-24 07:53:20,502 DEBUG [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-24 07:53:20,502 DEBUG [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99 2023-07-24 07:53:20,503 DEBUG [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:34211 2023-07-24 07:53:20,503 DEBUG [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-24 07:53:20,507 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 07:53:20,512 DEBUG [RS:2;jenkins-hbase4:37809] zookeeper.ZKUtil(162): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:20,512 WARN [RS:2;jenkins-hbase4:37809] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 07:53:20,512 INFO [RS:2;jenkins-hbase4:37809] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 07:53:20,512 DEBUG [RS:0;jenkins-hbase4:46877] zookeeper.ZKUtil(162): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:20,513 DEBUG [RS:1;jenkins-hbase4:46543] zookeeper.ZKUtil(162): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:20,513 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1948): logDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/WALs/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:20,513 WARN [RS:1;jenkins-hbase4:46543] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 07:53:20,513 WARN [RS:0;jenkins-hbase4:46877] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 07:53:20,513 INFO [RS:1;jenkins-hbase4:46543] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 07:53:20,514 INFO [RS:0;jenkins-hbase4:46877] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 07:53:20,514 DEBUG [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(1948): logDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/WALs/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:20,514 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46543,1690185199091] 2023-07-24 07:53:20,514 DEBUG [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(1948): logDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/WALs/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:20,514 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46877,1690185199045] 2023-07-24 07:53:20,514 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37809,1690185199136] 2023-07-24 07:53:20,525 DEBUG [RS:1;jenkins-hbase4:46543] zookeeper.ZKUtil(162): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:20,525 DEBUG [RS:0;jenkins-hbase4:46877] zookeeper.ZKUtil(162): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:20,525 DEBUG [RS:2;jenkins-hbase4:37809] zookeeper.ZKUtil(162): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:20,526 DEBUG [RS:1;jenkins-hbase4:46543] zookeeper.ZKUtil(162): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:20,526 DEBUG [RS:0;jenkins-hbase4:46877] zookeeper.ZKUtil(162): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:20,526 DEBUG [RS:2;jenkins-hbase4:37809] zookeeper.ZKUtil(162): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:20,526 DEBUG [RS:1;jenkins-hbase4:46543] zookeeper.ZKUtil(162): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:20,527 DEBUG [RS:0;jenkins-hbase4:46877] zookeeper.ZKUtil(162): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:20,527 DEBUG [RS:2;jenkins-hbase4:37809] zookeeper.ZKUtil(162): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:20,537 DEBUG [RS:1;jenkins-hbase4:46543] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 07:53:20,537 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 07:53:20,537 DEBUG [RS:0;jenkins-hbase4:46877] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 07:53:20,547 INFO [RS:1;jenkins-hbase4:46543] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 07:53:20,547 INFO [RS:2;jenkins-hbase4:37809] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 07:53:20,547 INFO [RS:0;jenkins-hbase4:46877] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 07:53:20,569 INFO [RS:2;jenkins-hbase4:37809] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 07:53:20,569 INFO [RS:0;jenkins-hbase4:46877] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 07:53:20,569 INFO [RS:1;jenkins-hbase4:46543] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 07:53:20,573 INFO [RS:2;jenkins-hbase4:37809] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 07:53:20,573 INFO [RS:0;jenkins-hbase4:46877] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 07:53:20,574 INFO [RS:2;jenkins-hbase4:37809] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,573 INFO [RS:1;jenkins-hbase4:46543] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 07:53:20,574 INFO [RS:0;jenkins-hbase4:46877] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,575 INFO [RS:1;jenkins-hbase4:46543] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,575 INFO [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 07:53:20,576 INFO [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 07:53:20,575 INFO [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 07:53:20,584 INFO [RS:0;jenkins-hbase4:46877] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,584 INFO [RS:2;jenkins-hbase4:37809] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,584 INFO [RS:1;jenkins-hbase4:46543] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,584 DEBUG [RS:0;jenkins-hbase4:46877] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,585 DEBUG [RS:2;jenkins-hbase4:37809] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,585 DEBUG [RS:0;jenkins-hbase4:46877] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,585 DEBUG [RS:1;jenkins-hbase4:46543] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:0;jenkins-hbase4:46877] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:2;jenkins-hbase4:37809] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:1;jenkins-hbase4:46543] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:0;jenkins-hbase4:46877] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:1;jenkins-hbase4:46543] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:2;jenkins-hbase4:37809] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:1;jenkins-hbase4:46543] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:0;jenkins-hbase4:46877] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:2;jenkins-hbase4:37809] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:0;jenkins-hbase4:46877] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 07:53:20,586 DEBUG [RS:2;jenkins-hbase4:37809] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:0;jenkins-hbase4:46877] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,586 DEBUG [RS:2;jenkins-hbase4:37809] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 07:53:20,586 DEBUG [RS:1;jenkins-hbase4:46543] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,587 DEBUG [RS:0;jenkins-hbase4:46877] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,587 DEBUG [RS:1;jenkins-hbase4:46543] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 07:53:20,587 DEBUG [RS:2;jenkins-hbase4:37809] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,587 DEBUG [RS:1;jenkins-hbase4:46543] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,587 DEBUG [RS:0;jenkins-hbase4:46877] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,587 DEBUG [RS:1;jenkins-hbase4:46543] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,587 DEBUG [RS:2;jenkins-hbase4:37809] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,587 DEBUG [RS:1;jenkins-hbase4:46543] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,587 DEBUG [RS:0;jenkins-hbase4:46877] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,587 DEBUG [RS:1;jenkins-hbase4:46543] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,587 DEBUG [RS:2;jenkins-hbase4:37809] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,588 DEBUG [RS:2;jenkins-hbase4:37809] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 07:53:20,593 INFO [RS:2;jenkins-hbase4:37809] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,593 INFO [RS:1;jenkins-hbase4:46543] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,593 INFO [RS:2;jenkins-hbase4:37809] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,593 INFO [RS:1;jenkins-hbase4:46543] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,593 INFO [RS:2;jenkins-hbase4:37809] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,593 INFO [RS:1;jenkins-hbase4:46543] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,593 INFO [RS:0;jenkins-hbase4:46877] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,593 INFO [RS:0;jenkins-hbase4:46877] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,593 INFO [RS:0;jenkins-hbase4:46877] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,612 INFO [RS:1;jenkins-hbase4:46543] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 07:53:20,612 INFO [RS:2;jenkins-hbase4:37809] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 07:53:20,612 INFO [RS:0;jenkins-hbase4:46877] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 07:53:20,615 INFO [RS:2;jenkins-hbase4:37809] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37809,1690185199136-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,615 INFO [RS:1;jenkins-hbase4:46543] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46543,1690185199091-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,615 INFO [RS:0;jenkins-hbase4:46877] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46877,1690185199045-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:20,633 DEBUG [jenkins-hbase4:37123] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-24 07:53:20,638 DEBUG [jenkins-hbase4:37123] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:53:20,638 INFO [RS:0;jenkins-hbase4:46877] regionserver.Replication(203): jenkins-hbase4.apache.org,46877,1690185199045 started 2023-07-24 07:53:20,639 INFO [RS:2;jenkins-hbase4:37809] regionserver.Replication(203): jenkins-hbase4.apache.org,37809,1690185199136 started 2023-07-24 07:53:20,639 INFO [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46877,1690185199045, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46877, sessionid=0x1019651e89e0001 2023-07-24 07:53:20,639 INFO [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,37809,1690185199136, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37809, sessionid=0x1019651e89e0003 2023-07-24 07:53:20,639 DEBUG [RS:2;jenkins-hbase4:37809] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 07:53:20,639 DEBUG [RS:0;jenkins-hbase4:46877] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 07:53:20,639 DEBUG [RS:2;jenkins-hbase4:37809] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:20,639 DEBUG [RS:0;jenkins-hbase4:46877] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:20,640 DEBUG [RS:2;jenkins-hbase4:37809] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37809,1690185199136' 2023-07-24 07:53:20,640 DEBUG [RS:0;jenkins-hbase4:46877] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46877,1690185199045' 2023-07-24 07:53:20,641 DEBUG [RS:2;jenkins-hbase4:37809] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:20,641 DEBUG [RS:0;jenkins-hbase4:46877] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:20,642 DEBUG [RS:0;jenkins-hbase4:46877] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:20,642 DEBUG [RS:2;jenkins-hbase4:37809] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:20,643 DEBUG [RS:2;jenkins-hbase4:37809] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 07:53:20,643 DEBUG [RS:0;jenkins-hbase4:46877] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 07:53:20,643 DEBUG [RS:2;jenkins-hbase4:37809] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 07:53:20,643 DEBUG [RS:0;jenkins-hbase4:46877] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 07:53:20,643 DEBUG [RS:2;jenkins-hbase4:37809] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:20,643 DEBUG [RS:2;jenkins-hbase4:37809] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37809,1690185199136' 2023-07-24 07:53:20,643 DEBUG [RS:2;jenkins-hbase4:37809] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 07:53:20,643 DEBUG [RS:0;jenkins-hbase4:46877] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:20,644 DEBUG [RS:0;jenkins-hbase4:46877] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46877,1690185199045' 2023-07-24 07:53:20,644 DEBUG [RS:0;jenkins-hbase4:46877] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 07:53:20,644 DEBUG [RS:2;jenkins-hbase4:37809] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 07:53:20,645 DEBUG [RS:0;jenkins-hbase4:46877] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 07:53:20,645 DEBUG [RS:2;jenkins-hbase4:37809] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 07:53:20,645 INFO [RS:2;jenkins-hbase4:37809] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 07:53:20,645 INFO [RS:2;jenkins-hbase4:37809] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 07:53:20,645 DEBUG [RS:0;jenkins-hbase4:46877] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 07:53:20,646 INFO [RS:0;jenkins-hbase4:46877] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 07:53:20,646 INFO [RS:0;jenkins-hbase4:46877] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 07:53:20,648 DEBUG [jenkins-hbase4:37123] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:53:20,648 DEBUG [jenkins-hbase4:37123] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:53:20,648 DEBUG [jenkins-hbase4:37123] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:53:20,648 DEBUG [jenkins-hbase4:37123] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:53:20,651 INFO [RS:1;jenkins-hbase4:46543] regionserver.Replication(203): jenkins-hbase4.apache.org,46543,1690185199091 started 2023-07-24 07:53:20,651 INFO [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46543,1690185199091, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46543, sessionid=0x1019651e89e0002 2023-07-24 07:53:20,651 DEBUG [RS:1;jenkins-hbase4:46543] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 07:53:20,651 DEBUG [RS:1;jenkins-hbase4:46543] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:20,651 DEBUG [RS:1;jenkins-hbase4:46543] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46543,1690185199091' 2023-07-24 07:53:20,651 DEBUG [RS:1;jenkins-hbase4:46543] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:20,652 DEBUG [RS:1;jenkins-hbase4:46543] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:20,653 DEBUG [RS:1;jenkins-hbase4:46543] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 07:53:20,653 DEBUG [RS:1;jenkins-hbase4:46543] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 07:53:20,653 DEBUG [RS:1;jenkins-hbase4:46543] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:20,653 DEBUG [RS:1;jenkins-hbase4:46543] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46543,1690185199091' 2023-07-24 07:53:20,653 DEBUG [RS:1;jenkins-hbase4:46543] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 07:53:20,653 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37809,1690185199136, state=OPENING 2023-07-24 07:53:20,654 DEBUG [RS:1;jenkins-hbase4:46543] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 07:53:20,654 DEBUG [RS:1;jenkins-hbase4:46543] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 07:53:20,654 INFO [RS:1;jenkins-hbase4:46543] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 07:53:20,654 INFO [RS:1;jenkins-hbase4:46543] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 07:53:20,663 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-24 07:53:20,664 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 07:53:20,665 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 07:53:20,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:53:20,757 INFO [RS:0;jenkins-hbase4:46877] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46877%2C1690185199045, suffix=, logDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/WALs/jenkins-hbase4.apache.org,46877,1690185199045, archiveDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/oldWALs, maxLogs=32 2023-07-24 07:53:20,757 INFO [RS:2;jenkins-hbase4:37809] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37809%2C1690185199136, suffix=, logDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/WALs/jenkins-hbase4.apache.org,37809,1690185199136, archiveDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/oldWALs, maxLogs=32 2023-07-24 07:53:20,758 INFO [RS:1;jenkins-hbase4:46543] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46543%2C1690185199091, suffix=, logDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/WALs/jenkins-hbase4.apache.org,46543,1690185199091, archiveDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/oldWALs, maxLogs=32 2023-07-24 07:53:20,782 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40639,DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed,DISK] 2023-07-24 07:53:20,782 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44119,DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7,DISK] 2023-07-24 07:53:20,784 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33477,DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa,DISK] 2023-07-24 07:53:20,794 INFO [RS:2;jenkins-hbase4:37809] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/WALs/jenkins-hbase4.apache.org,37809,1690185199136/jenkins-hbase4.apache.org%2C37809%2C1690185199136.1690185200762 2023-07-24 07:53:20,795 DEBUG [RS:2;jenkins-hbase4:37809] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44119,DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7,DISK], DatanodeInfoWithStorage[127.0.0.1:40639,DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed,DISK], DatanodeInfoWithStorage[127.0.0.1:33477,DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa,DISK]] 2023-07-24 07:53:20,808 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44119,DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7,DISK] 2023-07-24 07:53:20,810 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40639,DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed,DISK] 2023-07-24 07:53:20,810 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44119,DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7,DISK] 2023-07-24 07:53:20,810 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33477,DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa,DISK] 2023-07-24 07:53:20,811 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33477,DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa,DISK] 2023-07-24 07:53:20,811 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40639,DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed,DISK] 2023-07-24 07:53:20,823 INFO [RS:1;jenkins-hbase4:46543] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/WALs/jenkins-hbase4.apache.org,46543,1690185199091/jenkins-hbase4.apache.org%2C46543%2C1690185199091.1690185200762 2023-07-24 07:53:20,830 DEBUG [RS:1;jenkins-hbase4:46543] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40639,DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed,DISK], DatanodeInfoWithStorage[127.0.0.1:33477,DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa,DISK], DatanodeInfoWithStorage[127.0.0.1:44119,DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7,DISK]] 2023-07-24 07:53:20,834 INFO [RS:0;jenkins-hbase4:46877] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/WALs/jenkins-hbase4.apache.org,46877,1690185199045/jenkins-hbase4.apache.org%2C46877%2C1690185199045.1690185200762 2023-07-24 07:53:20,838 DEBUG [RS:0;jenkins-hbase4:46877] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44119,DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7,DISK], DatanodeInfoWithStorage[127.0.0.1:33477,DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa,DISK], DatanodeInfoWithStorage[127.0.0.1:40639,DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed,DISK]] 2023-07-24 07:53:20,857 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:20,860 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 07:53:20,864 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48846, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 07:53:20,878 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-24 07:53:20,879 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 07:53:20,883 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37809%2C1690185199136.meta, suffix=.meta, logDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/WALs/jenkins-hbase4.apache.org,37809,1690185199136, archiveDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/oldWALs, maxLogs=32 2023-07-24 07:53:20,906 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40639,DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed,DISK] 2023-07-24 07:53:20,907 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44119,DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7,DISK] 2023-07-24 07:53:20,909 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33477,DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa,DISK] 2023-07-24 07:53:20,917 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/WALs/jenkins-hbase4.apache.org,37809,1690185199136/jenkins-hbase4.apache.org%2C37809%2C1690185199136.meta.1690185200885.meta 2023-07-24 07:53:20,918 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40639,DS-9126357f-c5ac-4191-8ac2-8dcaac0b6fed,DISK], DatanodeInfoWithStorage[127.0.0.1:44119,DS-acb8e1a2-38af-4e6f-95a8-74b2241c68a7,DISK], DatanodeInfoWithStorage[127.0.0.1:33477,DS-0dc1e029-d0f9-4a8a-b396-7b3c11a460fa,DISK]] 2023-07-24 07:53:20,919 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:53:20,923 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:20,924 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-24 07:53:20,944 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-24 07:53:20,947 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-24 07:53:20,952 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-24 07:53:20,952 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:20,952 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-24 07:53:20,952 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-24 07:53:20,956 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-24 07:53:20,958 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/info 2023-07-24 07:53:20,958 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/info 2023-07-24 07:53:20,959 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-24 07:53:20,959 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:20,960 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-24 07:53:20,961 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/rep_barrier 2023-07-24 07:53:20,961 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/rep_barrier 2023-07-24 07:53:20,962 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-24 07:53:20,963 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:20,963 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-24 07:53:20,965 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/table 2023-07-24 07:53:20,965 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/table 2023-07-24 07:53:20,965 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-24 07:53:20,966 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:20,968 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740 2023-07-24 07:53:20,971 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740 2023-07-24 07:53:20,974 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-24 07:53:20,976 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-24 07:53:20,978 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70082671, jitterRate=0.04431317746639252}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-24 07:53:20,978 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-24 07:53:20,989 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1690185200849 2023-07-24 07:53:21,007 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-24 07:53:21,008 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-24 07:53:21,009 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37809,1690185199136, state=OPEN 2023-07-24 07:53:21,012 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-24 07:53:21,012 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 07:53:21,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-24 07:53:21,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37809,1690185199136 in 344 msec 2023-07-24 07:53:21,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-24 07:53:21,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 556 msec 2023-07-24 07:53:21,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 831 msec 2023-07-24 07:53:21,030 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690185201030, completionTime=-1 2023-07-24 07:53:21,030 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-07-24 07:53:21,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:21,101 DEBUG [hconnection-0x392b5020-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 07:53:21,104 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48852, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 07:53:21,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:21,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185261121 2023-07-24 07:53:21,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185321121 2023-07-24 07:53:21,121 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 90 msec 2023-07-24 07:53:21,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37123,1690185197841-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:21,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37123,1690185197841-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:21,149 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37123,1690185197841-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:21,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37123, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:21,152 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-24 07:53:21,163 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-24 07:53:21,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-24 07:53:21,172 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-24 07:53:21,182 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:21,184 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 07:53:21,187 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 07:53:21,211 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/hbase/namespace/61dcffa18562b861467de5a803476e2c 2023-07-24 07:53:21,213 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/hbase/namespace/61dcffa18562b861467de5a803476e2c empty. 2023-07-24 07:53:21,214 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/hbase/namespace/61dcffa18562b861467de5a803476e2c 2023-07-24 07:53:21,214 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-24 07:53:21,256 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-24 07:53:21,259 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 61dcffa18562b861467de5a803476e2c, NAME => 'hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp 2023-07-24 07:53:21,277 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:21,277 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 61dcffa18562b861467de5a803476e2c, disabling compactions & flushes 2023-07-24 07:53:21,277 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:53:21,278 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:53:21,278 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. after waiting 0 ms 2023-07-24 07:53:21,278 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:53:21,278 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:53:21,278 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 61dcffa18562b861467de5a803476e2c: 2023-07-24 07:53:21,283 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 07:53:21,298 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690185201285"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185201285"}]},"ts":"1690185201285"} 2023-07-24 07:53:21,327 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 07:53:21,329 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 07:53:21,334 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185201330"}]},"ts":"1690185201330"} 2023-07-24 07:53:21,339 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-24 07:53:21,345 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:53:21,346 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:53:21,346 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:53:21,346 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:53:21,346 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:53:21,349 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=61dcffa18562b861467de5a803476e2c, ASSIGN}] 2023-07-24 07:53:21,353 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=61dcffa18562b861467de5a803476e2c, ASSIGN 2023-07-24 07:53:21,355 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=61dcffa18562b861467de5a803476e2c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37809,1690185199136; forceNewPlan=false, retain=false 2023-07-24 07:53:21,508 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:53:21,509 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=61dcffa18562b861467de5a803476e2c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:21,510 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690185201509"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185201509"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185201509"}]},"ts":"1690185201509"} 2023-07-24 07:53:21,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 61dcffa18562b861467de5a803476e2c, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:53:21,674 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:53:21,674 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 61dcffa18562b861467de5a803476e2c, NAME => 'hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:53:21,676 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:21,676 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 61dcffa18562b861467de5a803476e2c 2023-07-24 07:53:21,677 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:21,677 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 61dcffa18562b861467de5a803476e2c 2023-07-24 07:53:21,677 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 61dcffa18562b861467de5a803476e2c 2023-07-24 07:53:21,679 INFO [StoreOpener-61dcffa18562b861467de5a803476e2c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 61dcffa18562b861467de5a803476e2c 2023-07-24 07:53:21,681 DEBUG [StoreOpener-61dcffa18562b861467de5a803476e2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/namespace/61dcffa18562b861467de5a803476e2c/info 2023-07-24 07:53:21,682 DEBUG [StoreOpener-61dcffa18562b861467de5a803476e2c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/namespace/61dcffa18562b861467de5a803476e2c/info 2023-07-24 07:53:21,682 INFO [StoreOpener-61dcffa18562b861467de5a803476e2c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 61dcffa18562b861467de5a803476e2c columnFamilyName info 2023-07-24 07:53:21,683 INFO [StoreOpener-61dcffa18562b861467de5a803476e2c-1] regionserver.HStore(310): Store=61dcffa18562b861467de5a803476e2c/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:21,685 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/namespace/61dcffa18562b861467de5a803476e2c 2023-07-24 07:53:21,686 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/namespace/61dcffa18562b861467de5a803476e2c 2023-07-24 07:53:21,690 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 61dcffa18562b861467de5a803476e2c 2023-07-24 07:53:21,693 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/namespace/61dcffa18562b861467de5a803476e2c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:53:21,694 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 61dcffa18562b861467de5a803476e2c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71519855, jitterRate=0.06572888791561127}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 07:53:21,694 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 61dcffa18562b861467de5a803476e2c: 2023-07-24 07:53:21,696 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c., pid=6, masterSystemTime=1690185201668 2023-07-24 07:53:21,700 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:53:21,700 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:53:21,702 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=61dcffa18562b861467de5a803476e2c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:21,702 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690185201701"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185201701"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185201701"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185201701"}]},"ts":"1690185201701"} 2023-07-24 07:53:21,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-24 07:53:21,710 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 61dcffa18562b861467de5a803476e2c, server=jenkins-hbase4.apache.org,37809,1690185199136 in 192 msec 2023-07-24 07:53:21,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-24 07:53:21,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=61dcffa18562b861467de5a803476e2c, ASSIGN in 362 msec 2023-07-24 07:53:21,716 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 07:53:21,716 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185201716"}]},"ts":"1690185201716"} 2023-07-24 07:53:21,718 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-24 07:53:21,721 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 07:53:21,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 548 msec 2023-07-24 07:53:21,784 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-07-24 07:53:21,786 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-07-24 07:53:21,787 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 07:53:21,820 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:21,838 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-24 07:53:21,845 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 32 msec 2023-07-24 07:53:21,854 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:21,867 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-24 07:53:21,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 19 msec 2023-07-24 07:53:21,885 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-24 07:53:21,887 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-24 07:53:21,887 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.685sec 2023-07-24 07:53:21,890 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-24 07:53:21,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-24 07:53:21,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-24 07:53:21,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37123,1690185197841-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-24 07:53:21,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37123,1690185197841-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-24 07:53:21,903 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-24 07:53:21,971 DEBUG [Listener at localhost/42537] zookeeper.ReadOnlyZKClient(139): Connect 0x72621703 to 127.0.0.1:55594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 07:53:21,978 DEBUG [Listener at localhost/42537] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7a41da2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-24 07:53:21,994 DEBUG [hconnection-0x58a9b8f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 07:53:22,006 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48864, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 07:53:22,017 INFO [Listener at localhost/42537] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37123,1690185197841 2023-07-24 07:53:22,051 INFO [Listener at localhost/42537] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=359, OpenFileDescriptor=694, MaxFileDescriptor=60000, SystemLoadAverage=345, ProcessCount=174, AvailableMemoryMB=9130 2023-07-24 07:53:22,056 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:22,065 DEBUG [Listener at localhost/42537] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-24 07:53:22,069 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54160, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-24 07:53:22,083 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-07-24 07:53:22,083 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 07:53:22,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 07:53:22,090 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,093 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 07:53:22,095 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 07:53:22,097 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:22,098 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5 empty. 2023-07-24 07:53:22,102 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:22,102 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-07-24 07:53:22,125 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-07-24 07:53:22,127 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => a9f3298786bf73c446662e1bb906c6e5, NAME => 'testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp 2023-07-24 07:53:22,151 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:22,152 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing a9f3298786bf73c446662e1bb906c6e5, disabling compactions & flushes 2023-07-24 07:53:22,152 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:22,152 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:22,152 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. after waiting 0 ms 2023-07-24 07:53:22,152 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:22,152 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:22,152 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for a9f3298786bf73c446662e1bb906c6e5: 2023-07-24 07:53:22,156 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 07:53:22,158 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690185202158"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185202158"}]},"ts":"1690185202158"} 2023-07-24 07:53:22,161 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 07:53:22,162 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 07:53:22,163 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185202163"}]},"ts":"1690185202163"} 2023-07-24 07:53:22,165 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-07-24 07:53:22,170 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:53:22,172 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:53:22,172 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:53:22,172 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:53:22,172 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:53:22,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9f3298786bf73c446662e1bb906c6e5, ASSIGN}] 2023-07-24 07:53:22,176 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9f3298786bf73c446662e1bb906c6e5, ASSIGN 2023-07-24 07:53:22,177 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9f3298786bf73c446662e1bb906c6e5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37809,1690185199136; forceNewPlan=false, retain=false 2023-07-24 07:53:22,327 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:53:22,329 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a9f3298786bf73c446662e1bb906c6e5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,329 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690185202328"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185202328"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185202328"}]},"ts":"1690185202328"} 2023-07-24 07:53:22,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure a9f3298786bf73c446662e1bb906c6e5, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:53:22,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:22,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a9f3298786bf73c446662e1bb906c6e5, NAME => 'testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:53:22,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:22,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:22,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:22,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:22,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:22,497 INFO [StoreOpener-a9f3298786bf73c446662e1bb906c6e5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:22,499 DEBUG [StoreOpener-a9f3298786bf73c446662e1bb906c6e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1 2023-07-24 07:53:22,500 DEBUG [StoreOpener-a9f3298786bf73c446662e1bb906c6e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1 2023-07-24 07:53:22,500 INFO [StoreOpener-a9f3298786bf73c446662e1bb906c6e5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9f3298786bf73c446662e1bb906c6e5 columnFamilyName cf1 2023-07-24 07:53:22,501 INFO [StoreOpener-a9f3298786bf73c446662e1bb906c6e5-1] regionserver.HStore(310): Store=a9f3298786bf73c446662e1bb906c6e5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:22,502 INFO [StoreOpener-a9f3298786bf73c446662e1bb906c6e5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:22,504 DEBUG [StoreOpener-a9f3298786bf73c446662e1bb906c6e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2 2023-07-24 07:53:22,504 DEBUG [StoreOpener-a9f3298786bf73c446662e1bb906c6e5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2 2023-07-24 07:53:22,505 INFO [StoreOpener-a9f3298786bf73c446662e1bb906c6e5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9f3298786bf73c446662e1bb906c6e5 columnFamilyName cf2 2023-07-24 07:53:22,505 INFO [StoreOpener-a9f3298786bf73c446662e1bb906c6e5-1] regionserver.HStore(310): Store=a9f3298786bf73c446662e1bb906c6e5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:22,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:22,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:22,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:22,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:22,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:53:22,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a9f3298786bf73c446662e1bb906c6e5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67563453, jitterRate=0.006773903965950012}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:22,519 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a9f3298786bf73c446662e1bb906c6e5: 2023-07-24 07:53:22,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5., pid=11, masterSystemTime=1690185202486 2023-07-24 07:53:22,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:22,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:22,525 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a9f3298786bf73c446662e1bb906c6e5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,525 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690185202525"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185202525"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185202525"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185202525"}]},"ts":"1690185202525"} 2023-07-24 07:53:22,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-24 07:53:22,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure a9f3298786bf73c446662e1bb906c6e5, server=jenkins-hbase4.apache.org,37809,1690185199136 in 196 msec 2023-07-24 07:53:22,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-24 07:53:22,536 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9f3298786bf73c446662e1bb906c6e5, ASSIGN in 360 msec 2023-07-24 07:53:22,537 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 07:53:22,537 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185202537"}]},"ts":"1690185202537"} 2023-07-24 07:53:22,540 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-07-24 07:53:22,543 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 07:53:22,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 455 msec 2023-07-24 07:53:22,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:22,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,626 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:22,627 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:22,627 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 07:53:22,627 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 07:53:22,628 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,628 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,630 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:22,630 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:22,630 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:22,630 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:22,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:22,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:22,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:22,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:22,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:22,630 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,630 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:22,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:22,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:22,632 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,632 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:22,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,632 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,634 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,634 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,634 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,634 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:22,634 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:22,634 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:22,634 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:22,635 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:22,634 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:22,635 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:22,635 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:22,636 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:22,638 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:22,638 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:22,638 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:22,638 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. started... 2023-07-24 07:53:22,639 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,640 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a9f3298786bf73c446662e1bb906c6e5 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 07:53:22,641 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,641 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,641 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:22,641 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,641 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:22,721 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/1727ed0d75b54fc2974373066d59b8b9 2023-07-24 07:53:22,790 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/01e8acff035c4f8099f478f93188752d 2023-07-24 07:53:22,801 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/1727ed0d75b54fc2974373066d59b8b9 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9 2023-07-24 07:53:22,811 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:53:22,814 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/01e8acff035c4f8099f478f93188752d as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d 2023-07-24 07:53:22,824 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:53:22,826 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for a9f3298786bf73c446662e1bb906c6e5 in 186ms, sequenceid=6, compaction requested=false 2023-07-24 07:53:22,828 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:22,831 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a9f3298786bf73c446662e1bb906c6e5: 2023-07-24 07:53:22,831 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:22,831 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:22,832 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:22,832 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,832 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:22,832 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:22,834 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,834 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:22,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:22,835 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,835 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:22,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:22,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:22,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:22,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:22,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:22,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 07:53:22,838 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:22,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53e48d9e[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:22,838 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,844 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,844 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,844 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:22,845 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:22,845 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:22,846 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:22,851 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:22,851 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:22,851 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:22,853 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:22,853 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:22,853 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:22,854 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,854 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:22,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:22,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:22,854 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:22,854 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:22,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:22,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:22,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:22,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:22,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:22,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:22,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:22,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37809,1690185199136': 2023-07-24 07:53:22,861 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 07:53:22,861 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:22,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:22,861 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:22,861 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,861 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:22,864 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,864 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,864 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:22,864 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,864 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,864 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:22,864 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:22,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:22,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:22,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:22,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:22,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:22,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:22,865 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:22,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:22,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:22,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:22,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,866 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:22,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,867 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:22,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:22,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:22,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:22,870 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:22,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,870 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:22,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:22,871 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:22,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,882 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:22,882 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:22,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:22,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:22,883 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:22,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:22,882 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,883 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,882 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:22,883 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:22,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:22,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:22,883 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:22,883 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:22,883 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,884 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:22,884 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,884 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:22,884 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:22,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:22,883 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:22,884 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:22,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:22,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:22,883 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:22,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:22,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:22,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:22,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:22,884 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:22,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:22,887 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:22,887 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:22,987 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:22,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:23,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:23,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:23,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,025 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:23,025 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,026 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 07:53:23,026 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 07:53:23,027 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,027 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,028 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,028 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,028 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,028 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,029 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,029 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:23,029 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,029 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,030 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,030 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,030 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,030 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,030 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,030 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,030 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,031 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,031 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,031 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,031 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,031 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,031 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,031 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,032 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,031 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,032 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,032 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,033 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,032 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,032 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,033 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,033 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,033 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:23,033 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:23,034 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:23,034 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. started... 2023-07-24 07:53:23,034 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a9f3298786bf73c446662e1bb906c6e5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:23,036 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,036 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,036 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,036 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,036 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,036 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,058 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/fa761475e42e4086a390c3a5d9ace2fb 2023-07-24 07:53:23,088 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/c0280b228cb94c9d94d02ba658f51355 2023-07-24 07:53:23,098 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/fa761475e42e4086a390c3a5d9ace2fb as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb 2023-07-24 07:53:23,108 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:53:23,109 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/c0280b228cb94c9d94d02ba658f51355 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355 2023-07-24 07:53:23,119 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:53:23,120 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a9f3298786bf73c446662e1bb906c6e5 in 86ms, sequenceid=19, compaction requested=false 2023-07-24 07:53:23,120 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a9f3298786bf73c446662e1bb906c6e5: 2023-07-24 07:53:23,120 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:23,120 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,121 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,123 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,123 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,123 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,124 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 07:53:23,127 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5cf3903[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:23,127 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:23,127 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,129 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,129 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,129 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,130 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,130 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,130 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,130 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:23,130 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,130 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,130 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,130 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:23,130 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:23,130 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,130 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:23,132 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,132 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,132 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,133 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,133 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,133 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,134 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,133 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,134 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,134 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37809,1690185199136': 2023-07-24 07:53:23,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:23,140 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 07:53:23,140 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:23,140 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:23,140 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,140 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:23,142 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,142 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,142 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,142 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,143 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,143 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,142 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,144 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,144 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,145 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,148 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,148 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,149 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,161 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,161 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,161 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,161 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,161 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,161 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,161 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,161 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,162 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,161 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:23,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:23,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,162 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,163 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,163 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,163 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,163 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,163 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,163 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,163 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:23,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,163 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:23,163 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,263 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:23,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:23,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:23,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:23,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,293 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:23,293 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,294 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 07:53:23,294 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 07:53:23,295 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,295 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,296 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,296 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,296 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,296 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,297 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,297 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,299 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,299 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,299 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,299 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,299 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,299 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,299 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,299 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,302 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,302 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,302 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,302 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,301 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,302 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,302 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,302 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,302 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,302 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,302 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,303 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,303 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,303 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:23,303 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:23,303 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:23,303 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. started... 2023-07-24 07:53:23,304 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a9f3298786bf73c446662e1bb906c6e5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:23,305 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,305 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,306 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,306 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,306 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,306 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,324 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/b571dc026b1d4fa29d46cc474574027a 2023-07-24 07:53:23,348 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/220c737baeb7466fb9b74acc451fa4a1 2023-07-24 07:53:23,360 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/b571dc026b1d4fa29d46cc474574027a as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a 2023-07-24 07:53:23,368 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:53:23,370 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/220c737baeb7466fb9b74acc451fa4a1 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1 2023-07-24 07:53:23,379 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:53:23,380 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a9f3298786bf73c446662e1bb906c6e5 in 76ms, sequenceid=32, compaction requested=true 2023-07-24 07:53:23,380 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a9f3298786bf73c446662e1bb906c6e5: 2023-07-24 07:53:23,380 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:23,380 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:23,381 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:23,381 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,381 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,381 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,384 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,384 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,385 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,385 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,388 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 07:53:23,388 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53c4b57b[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:23,388 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:23,388 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,389 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,389 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,389 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,389 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,389 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,390 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,390 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:23,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,389 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:23,390 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,390 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,390 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:23,390 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,390 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,390 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:23,393 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,393 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,393 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,394 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,394 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,394 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,394 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,395 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,395 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,395 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37809,1690185199136': 2023-07-24 07:53:23,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:23,402 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 07:53:23,402 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:23,402 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:23,402 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,402 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:23,405 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,405 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,405 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,405 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,405 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,406 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,406 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,406 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,408 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,408 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,411 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,412 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,412 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,416 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,416 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,416 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,416 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,416 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,416 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,416 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,416 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,416 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,416 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:23,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:23,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,416 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,417 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,417 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,417 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,417 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:23,418 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:23,417 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,418 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,418 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,418 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,518 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:23,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:23,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:23,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:23,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,546 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:23,546 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,546 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 07:53:23,546 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 07:53:23,547 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,547 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,549 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,549 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,549 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,549 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,549 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,549 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:23,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,550 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,550 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,550 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,551 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,551 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,551 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,551 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,551 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,552 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,552 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,551 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,552 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,552 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,553 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,553 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,553 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,553 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,553 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,554 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,554 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,554 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,554 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:23,553 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:23,554 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:23,554 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. started... 2023-07-24 07:53:23,555 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a9f3298786bf73c446662e1bb906c6e5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:23,556 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,558 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,558 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,558 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,559 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,559 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,581 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/3b09af552d514727a84ae414de420a8e 2023-07-24 07:53:23,612 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/0b9ea7e45d864b29bf075937fd6b2577 2023-07-24 07:53:23,621 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/3b09af552d514727a84ae414de420a8e as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e 2023-07-24 07:53:23,629 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:53:23,631 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/0b9ea7e45d864b29bf075937fd6b2577 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577 2023-07-24 07:53:23,639 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:53:23,640 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a9f3298786bf73c446662e1bb906c6e5 in 85ms, sequenceid=45, compaction requested=true 2023-07-24 07:53:23,640 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a9f3298786bf73c446662e1bb906c6e5: 2023-07-24 07:53:23,640 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:23,640 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:23,641 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:23,641 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,641 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,641 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,646 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,646 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,647 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,647 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 07:53:23,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ec07545[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:23,649 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:23,649 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,651 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,651 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,651 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,651 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,651 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,651 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:23,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,651 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,651 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:23,651 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,652 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,652 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,652 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,652 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:23,652 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:23,658 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,658 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,658 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,658 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,659 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,659 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,659 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,659 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,659 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,660 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37809,1690185199136': 2023-07-24 07:53:23,665 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:23,665 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 07:53:23,665 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:23,665 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:23,665 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,665 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:23,667 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,667 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,667 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,667 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,667 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,667 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,668 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,667 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,668 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,669 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,671 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,671 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,672 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,675 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,675 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,676 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,676 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,676 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,675 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,676 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:23,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:23,676 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,677 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,677 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,677 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,677 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:23,677 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,678 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:23,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,678 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,678 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,678 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,678 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,678 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,678 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,778 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:23,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:23,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:23,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:23,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,805 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:23,806 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,806 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 07:53:23,806 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 07:53:23,807 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,807 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,810 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,810 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,810 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,810 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,811 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,811 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:23,811 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,811 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,811 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,812 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,812 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,812 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,812 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,813 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,813 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,814 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,814 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,814 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,814 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,814 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,814 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,814 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,814 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:23,815 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,815 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,815 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:23,815 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:23,815 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:23,820 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:23,820 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:23,820 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. started... 2023-07-24 07:53:23,820 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:23,820 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,820 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,820 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a9f3298786bf73c446662e1bb906c6e5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:23,822 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,823 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,823 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,823 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,824 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,824 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,840 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/9db03282f75a467a80913c9104b71cc3 2023-07-24 07:53:23,868 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/1301c11c17f74693ac13c654e3eb2939 2023-07-24 07:53:23,877 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/9db03282f75a467a80913c9104b71cc3 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/9db03282f75a467a80913c9104b71cc3 2023-07-24 07:53:23,885 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/9db03282f75a467a80913c9104b71cc3, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:53:23,887 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/1301c11c17f74693ac13c654e3eb2939 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/1301c11c17f74693ac13c654e3eb2939 2023-07-24 07:53:23,895 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/1301c11c17f74693ac13c654e3eb2939, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:53:23,896 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a9f3298786bf73c446662e1bb906c6e5 in 76ms, sequenceid=58, compaction requested=true 2023-07-24 07:53:23,896 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a9f3298786bf73c446662e1bb906c6e5: 2023-07-24 07:53:23,896 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:23,896 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:23,897 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:23,897 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,897 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:23,897 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:23,899 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,899 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,900 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,900 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:23,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 07:53:23,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9bf9f7e[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:23,902 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:23,902 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,905 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,905 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,905 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,905 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,905 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,905 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,905 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:23,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,905 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:23,905 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,905 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,905 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:23,905 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:23,905 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:23,905 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:23,907 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,907 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,907 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,908 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,908 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,908 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:23,908 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,908 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,908 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,909 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:23,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37809,1690185199136': 2023-07-24 07:53:23,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:23,914 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 07:53:23,915 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:23,915 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:23,915 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,915 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:23,916 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,916 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,917 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,917 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,916 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,916 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:23,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,917 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:23,917 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,918 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,919 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:23,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,921 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:23,921 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,921 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,928 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,928 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,928 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,928 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,928 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:23,929 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:23,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:23,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,930 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:23,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:23,930 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:23,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:23,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,930 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,931 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:23,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,931 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,931 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:23,931 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:23,931 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:23,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:23,931 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:23,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:23,931 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:23,931 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,032 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:24,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:24,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:24,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:24,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,057 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:24,058 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:24,058 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 07:53:24,058 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 07:53:24,059 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,059 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,060 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,060 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:24,060 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:24,060 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:24,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:24,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:24,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:24,061 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,061 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:24,062 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:24,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:24,063 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:24,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 07:53:24,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,063 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:24,063 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 07:53:24,063 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:24,063 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:24,063 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:24,064 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:24,065 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:24,065 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:24,064 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 07:53:24,065 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:24,065 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:24,065 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:24,065 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:24,065 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:24,065 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:24,065 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. started... 2023-07-24 07:53:24,065 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a9f3298786bf73c446662e1bb906c6e5 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:24,066 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,066 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,067 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,067 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:24,067 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,067 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:24,082 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/1365d727c5024f168b214733b173f0c3 2023-07-24 07:53:24,108 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/6117bbd26e9e44f884a166f2fbff5bd1 2023-07-24 07:53:24,117 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/1365d727c5024f168b214733b173f0c3 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1365d727c5024f168b214733b173f0c3 2023-07-24 07:53:24,125 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1365d727c5024f168b214733b173f0c3, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:53:24,126 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/6117bbd26e9e44f884a166f2fbff5bd1 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/6117bbd26e9e44f884a166f2fbff5bd1 2023-07-24 07:53:24,135 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/6117bbd26e9e44f884a166f2fbff5bd1, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:53:24,137 INFO [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a9f3298786bf73c446662e1bb906c6e5 in 72ms, sequenceid=71, compaction requested=true 2023-07-24 07:53:24,137 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a9f3298786bf73c446662e1bb906c6e5: 2023-07-24 07:53:24,137 DEBUG [rs(jenkins-hbase4.apache.org,37809,1690185199136)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:24,137 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:24,139 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:24,139 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,139 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 07:53:24,139 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 07:53:24,141 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,141 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:24,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:24,142 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,142 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:24,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:24,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:24,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:24,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:24,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:24,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 07:53:24,145 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:24,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75c4b86b[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:24,145 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,150 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,150 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,150 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,151 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:24,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,151 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,151 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:24,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,151 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:24,151 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 07:53:24,151 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:24,151 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:24,151 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:24,151 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:24,151 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 07:53:24,151 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:24,153 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:24,153 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:24,153 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:24,154 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,154 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:24,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,154 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:24,154 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 07:53:24,154 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:24,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:24,155 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:24,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:24,155 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 07:53:24,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:24,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:24,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:24,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:24,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:24,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:24,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:24,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37809,1690185199136': 2023-07-24 07:53:24,160 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:24,160 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 07:53:24,160 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:24,160 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:24,160 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,160 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:24,162 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,162 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:24,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,162 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,162 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:24,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,162 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,162 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:24,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,162 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:24,162 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:24,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:24,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:24,163 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:24,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,163 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:24,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:24,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:24,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,166 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:24,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:24,166 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:24,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:24,166 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:24,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:24,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:24,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:24,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:24,170 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:24,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:24,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:24,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:24,171 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:24,171 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:24,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 07:53:24,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:24,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:24,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:24,172 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,172 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:24,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:24,172 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:24,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:24,172 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:24,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:24,172 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:24,172 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 07:53:24,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:24,172 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:24,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:24,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:24,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:24,273 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:24,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:24,297 DEBUG [Listener at localhost/42537] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,304 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:24,305 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:24,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:24,306 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:24,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9f3298786bf73c446662e1bb906c6e5, UNASSIGN}] 2023-07-24 07:53:24,315 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9f3298786bf73c446662e1bb906c6e5, UNASSIGN 2023-07-24 07:53:24,316 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a9f3298786bf73c446662e1bb906c6e5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:24,317 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690185204316"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185204316"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185204316"}]},"ts":"1690185204316"} 2023-07-24 07:53:24,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure a9f3298786bf73c446662e1bb906c6e5, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:53:24,483 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,484 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a9f3298786bf73c446662e1bb906c6e5, disabling compactions & flushes 2023-07-24 07:53:24,484 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:24,484 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:24,484 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. after waiting 0 ms 2023-07-24 07:53:24,484 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:24,484 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing a9f3298786bf73c446662e1bb906c6e5 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 07:53:24,505 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/c2eec162c33140bcb05144ddcc92623a 2023-07-24 07:53:24,533 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/a203a2c08df540e5816c328c5236de43 2023-07-24 07:53:24,541 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf1/c2eec162c33140bcb05144ddcc92623a as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/c2eec162c33140bcb05144ddcc92623a 2023-07-24 07:53:24,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/c2eec162c33140bcb05144ddcc92623a, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:53:24,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.tmp/cf2/a203a2c08df540e5816c328c5236de43 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/a203a2c08df540e5816c328c5236de43 2023-07-24 07:53:24,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/a203a2c08df540e5816c328c5236de43, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:53:24,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a9f3298786bf73c446662e1bb906c6e5 in 79ms, sequenceid=82, compaction requested=true 2023-07-24 07:53:24,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 07:53:24,587 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:24,588 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. 2023-07-24 07:53:24,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a9f3298786bf73c446662e1bb906c6e5: 2023-07-24 07:53:24,591 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,592 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=a9f3298786bf73c446662e1bb906c6e5, regionState=CLOSED 2023-07-24 07:53:24,593 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690185204592"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185204592"}]},"ts":"1690185204592"} 2023-07-24 07:53:24,597 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-24 07:53:24,597 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure a9f3298786bf73c446662e1bb906c6e5, server=jenkins-hbase4.apache.org,37809,1690185199136 in 276 msec 2023-07-24 07:53:24,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-24 07:53:24,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=a9f3298786bf73c446662e1bb906c6e5, UNASSIGN in 284 msec 2023-07-24 07:53:24,623 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=a9f3298786bf73c446662e1bb906c6e5, threads=14 2023-07-24 07:53:24,624 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,624 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,626 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/1301c11c17f74693ac13c654e3eb2939 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,626 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,626 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/6117bbd26e9e44f884a166f2fbff5bd1 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,627 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/a203a2c08df540e5816c328c5236de43 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,627 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,628 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1365d727c5024f168b214733b173f0c3 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,628 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,628 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,628 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/9db03282f75a467a80913c9104b71cc3 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,628 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,629 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/c2eec162c33140bcb05144ddcc92623a for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,629 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,652 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/c2eec162c33140bcb05144ddcc92623a, top=true 2023-07-24 07:53:24,655 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/6117bbd26e9e44f884a166f2fbff5bd1, top=true 2023-07-24 07:53:24,678 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1365d727c5024f168b214733b173f0c3, top=true 2023-07-24 07:53:24,682 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/9db03282f75a467a80913c9104b71cc3, top=true 2023-07-24 07:53:24,682 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/1301c11c17f74693ac13c654e3eb2939, top=true 2023-07-24 07:53:24,688 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/a203a2c08df540e5816c328c5236de43, top=true 2023-07-24 07:53:24,734 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.splits/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-6117bbd26e9e44f884a166f2fbff5bd1 for child: 542548d310d66672741e6f9e560f852c, parent: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,734 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/6117bbd26e9e44f884a166f2fbff5bd1 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,743 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.splits/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1301c11c17f74693ac13c654e3eb2939 for child: 542548d310d66672741e6f9e560f852c, parent: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,743 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/1301c11c17f74693ac13c654e3eb2939 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,744 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.splits/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-c2eec162c33140bcb05144ddcc92623a for child: 542548d310d66672741e6f9e560f852c, parent: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,744 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/c2eec162c33140bcb05144ddcc92623a for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,747 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.splits/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-a203a2c08df540e5816c328c5236de43 for child: 542548d310d66672741e6f9e560f852c, parent: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,747 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/a203a2c08df540e5816c328c5236de43 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,751 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.splits/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-9db03282f75a467a80913c9104b71cc3 for child: 542548d310d66672741e6f9e560f852c, parent: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,751 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/9db03282f75a467a80913c9104b71cc3 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,751 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/.splits/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1365d727c5024f168b214733b173f0c3 for child: 542548d310d66672741e6f9e560f852c, parent: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,751 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1365d727c5024f168b214733b173f0c3 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,858 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,877 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,885 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,911 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,917 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,918 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9 for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,923 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,929 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e for region: a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:24,930 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region a9f3298786bf73c446662e1bb906c6e5 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-07-24 07:53:24,992 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:53:24,995 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:53:24,997 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690185204997"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1690185204997"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1690185204997"}]},"ts":"1690185204997"} 2023-07-24 07:53:24,998 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690185204997"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185204997"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185204997"}]},"ts":"1690185204997"} 2023-07-24 07:53:24,998 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690185204997"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185204997"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185204997"}]},"ts":"1690185204997"} 2023-07-24 07:53:25,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5eca7056352c3c5c9619dde691b5b9b3, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=542548d310d66672741e6f9e560f852c, ASSIGN}] 2023-07-24 07:53:25,050 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=542548d310d66672741e6f9e560f852c, ASSIGN 2023-07-24 07:53:25,051 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=542548d310d66672741e6f9e560f852c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37809,1690185199136; forceNewPlan=false, retain=false 2023-07-24 07:53:25,052 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5eca7056352c3c5c9619dde691b5b9b3, ASSIGN 2023-07-24 07:53:25,052 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5eca7056352c3c5c9619dde691b5b9b3, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37809,1690185199136; forceNewPlan=false, retain=false 2023-07-24 07:53:25,201 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 07:53:25,203 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5eca7056352c3c5c9619dde691b5b9b3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:25,203 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690185205203"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185205203"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185205203"}]},"ts":"1690185205203"} 2023-07-24 07:53:25,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 5eca7056352c3c5c9619dde691b5b9b3, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:53:25,209 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=542548d310d66672741e6f9e560f852c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:25,210 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690185205209"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185205209"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185205209"}]},"ts":"1690185205209"} 2023-07-24 07:53:25,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 542548d310d66672741e6f9e560f852c, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:53:25,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. 2023-07-24 07:53:25,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5eca7056352c3c5c9619dde691b5b9b3, NAME => 'testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.', STARTKEY => '', ENDKEY => '26'} 2023-07-24 07:53:25,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:25,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:25,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:25,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:25,364 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:25,366 INFO [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:25,367 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1 2023-07-24 07:53:25,367 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1 2023-07-24 07:53:25,368 INFO [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5eca7056352c3c5c9619dde691b5b9b3 columnFamilyName cf1 2023-07-24 07:53:25,402 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9-bottom 2023-07-24 07:53:25,410 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e-bottom 2023-07-24 07:53:25,418 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a-bottom 2023-07-24 07:53:25,425 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb-bottom 2023-07-24 07:53:25,426 INFO [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(310): Store=5eca7056352c3c5c9619dde691b5b9b3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:25,426 INFO [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:25,427 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2 2023-07-24 07:53:25,427 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2 2023-07-24 07:53:25,428 INFO [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5eca7056352c3c5c9619dde691b5b9b3 columnFamilyName cf2 2023-07-24 07:53:25,445 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d-bottom 2023-07-24 07:53:25,453 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577-bottom 2023-07-24 07:53:25,461 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1-bottom 2023-07-24 07:53:25,468 DEBUG [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355-bottom 2023-07-24 07:53:25,468 INFO [StoreOpener-5eca7056352c3c5c9619dde691b5b9b3-1] regionserver.HStore(310): Store=5eca7056352c3c5c9619dde691b5b9b3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:25,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:25,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:25,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:25,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:25,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5eca7056352c3c5c9619dde691b5b9b3; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75159803, jitterRate=0.11996833980083466}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:25,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5eca7056352c3c5c9619dde691b5b9b3: 2023-07-24 07:53:25,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3., pid=17, masterSystemTime=1690185205358 2023-07-24 07:53:25,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:25,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 07:53:25,481 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-07-24 07:53:25,481 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-07-24 07:53:25,483 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. 2023-07-24 07:53:25,483 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. 2023-07-24 07:53:25,483 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HStore(1912): 5eca7056352c3c5c9619dde691b5b9b3/cf1 is initiating minor compaction (all files) 2023-07-24 07:53:25,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. 2023-07-24 07:53:25,483 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. 2023-07-24 07:53:25,483 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HStore(1912): 5eca7056352c3c5c9619dde691b5b9b3/cf2 is initiating minor compaction (all files) 2023-07-24 07:53:25,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. 2023-07-24 07:53:25,483 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 5eca7056352c3c5c9619dde691b5b9b3/cf2 in testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. 2023-07-24 07:53:25,483 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5eca7056352c3c5c9619dde691b5b9b3/cf1 in testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. 2023-07-24 07:53:25,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 542548d310d66672741e6f9e560f852c, NAME => 'testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.', STARTKEY => '26', ENDKEY => ''} 2023-07-24 07:53:25,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:25,484 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5eca7056352c3c5c9619dde691b5b9b3, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:25,484 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e-bottom] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/.tmp, totalSize=19.9 K 2023-07-24 07:53:25,484 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577-bottom] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/.tmp, totalSize=19.9 K 2023-07-24 07:53:25,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 542548d310d66672741e6f9e560f852c 2023-07-24 07:53:25,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:25,485 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690185205484"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185205484"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185205484"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185205484"}]},"ts":"1690185205484"} 2023-07-24 07:53:25,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 542548d310d66672741e6f9e560f852c 2023-07-24 07:53:25,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 542548d310d66672741e6f9e560f852c 2023-07-24 07:53:25,487 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.Compactor(207): Compacting 01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185202597 2023-07-24 07:53:25,488 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.Compactor(207): Compacting 1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185202597 2023-07-24 07:53:25,489 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.Compactor(207): Compacting c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185202997 2023-07-24 07:53:25,489 INFO [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 542548d310d66672741e6f9e560f852c 2023-07-24 07:53:25,489 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.Compactor(207): Compacting fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185202997 2023-07-24 07:53:25,490 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.Compactor(207): Compacting 220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185203266 2023-07-24 07:53:25,490 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.Compactor(207): Compacting b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185203266 2023-07-24 07:53:25,490 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.Compactor(207): Compacting 0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185203520 2023-07-24 07:53:25,492 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1 2023-07-24 07:53:25,492 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1 2023-07-24 07:53:25,492 INFO [StoreOpener-542548d310d66672741e6f9e560f852c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 542548d310d66672741e6f9e560f852c columnFamilyName cf1 2023-07-24 07:53:25,493 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.Compactor(207): Compacting 3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185203520 2023-07-24 07:53:25,494 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-07-24 07:53:25,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 5eca7056352c3c5c9619dde691b5b9b3, server=jenkins-hbase4.apache.org,37809,1690185199136 in 283 msec 2023-07-24 07:53:25,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5eca7056352c3c5c9619dde691b5b9b3, ASSIGN in 447 msec 2023-07-24 07:53:25,524 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9-top 2023-07-24 07:53:25,525 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] throttle.PressureAwareThroughputController(145): 5eca7056352c3c5c9619dde691b5b9b3#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:25,525 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5eca7056352c3c5c9619dde691b5b9b3#cf1#compaction#15 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:25,539 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e-top 2023-07-24 07:53:25,546 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a-top 2023-07-24 07:53:25,554 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb-top 2023-07-24 07:53:25,565 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1365d727c5024f168b214733b173f0c3 2023-07-24 07:53:25,572 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-9db03282f75a467a80913c9104b71cc3 2023-07-24 07:53:25,578 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-c2eec162c33140bcb05144ddcc92623a 2023-07-24 07:53:25,578 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/.tmp/cf1/72c74ba0e1d14e9898e187f32269d07a as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/72c74ba0e1d14e9898e187f32269d07a 2023-07-24 07:53:25,580 INFO [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(310): Store=542548d310d66672741e6f9e560f852c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:25,580 INFO [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 542548d310d66672741e6f9e560f852c 2023-07-24 07:53:25,580 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/.tmp/cf2/6fb00563aba64b6e99d6ca5607e34866 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/6fb00563aba64b6e99d6ca5607e34866 2023-07-24 07:53:25,582 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2 2023-07-24 07:53:25,582 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2 2023-07-24 07:53:25,582 INFO [StoreOpener-542548d310d66672741e6f9e560f852c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 542548d310d66672741e6f9e560f852c columnFamilyName cf2 2023-07-24 07:53:25,601 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5eca7056352c3c5c9619dde691b5b9b3/cf2 of 5eca7056352c3c5c9619dde691b5b9b3 into 6fb00563aba64b6e99d6ca5607e34866(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:25,603 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5eca7056352c3c5c9619dde691b5b9b3: 2023-07-24 07:53:25,604 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3., storeName=5eca7056352c3c5c9619dde691b5b9b3/cf2, priority=12, startTime=1690185205479; duration=0sec 2023-07-24 07:53:25,601 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5eca7056352c3c5c9619dde691b5b9b3/cf1 of 5eca7056352c3c5c9619dde691b5b9b3 into 72c74ba0e1d14e9898e187f32269d07a(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:25,605 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5eca7056352c3c5c9619dde691b5b9b3: 2023-07-24 07:53:25,605 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3., storeName=5eca7056352c3c5c9619dde691b5b9b3/cf1, priority=12, startTime=1690185205479; duration=0sec 2023-07-24 07:53:25,606 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:25,606 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:25,613 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d-top 2023-07-24 07:53:25,618 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577-top 2023-07-24 07:53:25,623 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1-top 2023-07-24 07:53:25,628 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355-top 2023-07-24 07:53:25,633 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1301c11c17f74693ac13c654e3eb2939 2023-07-24 07:53:25,637 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-6117bbd26e9e44f884a166f2fbff5bd1 2023-07-24 07:53:25,642 DEBUG [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-a203a2c08df540e5816c328c5236de43 2023-07-24 07:53:25,642 INFO [StoreOpener-542548d310d66672741e6f9e560f852c-1] regionserver.HStore(310): Store=542548d310d66672741e6f9e560f852c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:25,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c 2023-07-24 07:53:25,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c 2023-07-24 07:53:25,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:25,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 542548d310d66672741e6f9e560f852c 2023-07-24 07:53:25,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 542548d310d66672741e6f9e560f852c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73775450, jitterRate=0.09933987259864807}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:25,652 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 542548d310d66672741e6f9e560f852c: 2023-07-24 07:53:25,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c., pid=18, masterSystemTime=1690185205358 2023-07-24 07:53:25,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:25,654 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:25,654 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:25,654 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:25,654 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. 2023-07-24 07:53:25,654 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. 2023-07-24 07:53:25,655 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HStore(1912): 542548d310d66672741e6f9e560f852c/cf1 is initiating minor compaction (all files) 2023-07-24 07:53:25,655 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HStore(1912): 542548d310d66672741e6f9e560f852c/cf2 is initiating minor compaction (all files) 2023-07-24 07:53:25,655 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 542548d310d66672741e6f9e560f852c/cf1 in testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. 2023-07-24 07:53:25,655 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 542548d310d66672741e6f9e560f852c/cf2 in testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. 2023-07-24 07:53:25,655 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-9db03282f75a467a80913c9104b71cc3, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1365d727c5024f168b214733b173f0c3, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-c2eec162c33140bcb05144ddcc92623a] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/.tmp, totalSize=34.9 K 2023-07-24 07:53:25,655 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1301c11c17f74693ac13c654e3eb2939, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-6117bbd26e9e44f884a166f2fbff5bd1, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-a203a2c08df540e5816c328c5236de43] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/.tmp, totalSize=34.9 K 2023-07-24 07:53:25,657 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.Compactor(207): Compacting 1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690185202597 2023-07-24 07:53:25,659 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.Compactor(207): Compacting fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690185202997 2023-07-24 07:53:25,659 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.Compactor(207): Compacting 01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690185202597 2023-07-24 07:53:25,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. 2023-07-24 07:53:25,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. 2023-07-24 07:53:25,661 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.Compactor(207): Compacting c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690185202997 2023-07-24 07:53:25,662 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=542548d310d66672741e6f9e560f852c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:25,662 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690185205662"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185205662"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185205662"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185205662"}]},"ts":"1690185205662"} 2023-07-24 07:53:25,662 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.Compactor(207): Compacting b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690185203266 2023-07-24 07:53:25,663 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.Compactor(207): Compacting 220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690185203266 2023-07-24 07:53:25,666 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.Compactor(207): Compacting 0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690185203520 2023-07-24 07:53:25,666 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.Compactor(207): Compacting 3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690185203520 2023-07-24 07:53:25,666 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1301c11c17f74693ac13c654e3eb2939, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185203781 2023-07-24 07:53:25,667 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-9db03282f75a467a80913c9104b71cc3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185203781 2023-07-24 07:53:25,667 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-6117bbd26e9e44f884a166f2fbff5bd1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185204035 2023-07-24 07:53:25,668 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1365d727c5024f168b214733b173f0c3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185204035 2023-07-24 07:53:25,668 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-a203a2c08df540e5816c328c5236de43, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185204276 2023-07-24 07:53:25,669 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-c2eec162c33140bcb05144ddcc92623a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185204276 2023-07-24 07:53:25,671 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-07-24 07:53:25,671 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 542548d310d66672741e6f9e560f852c, server=jenkins-hbase4.apache.org,37809,1690185199136 in 455 msec 2023-07-24 07:53:25,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-07-24 07:53:25,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=542548d310d66672741e6f9e560f852c, ASSIGN in 623 msec 2023-07-24 07:53:25,680 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c in 1.3800 sec 2023-07-24 07:53:25,700 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] throttle.PressureAwareThroughputController(145): 542548d310d66672741e6f9e560f852c#cf1#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:25,700 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] throttle.PressureAwareThroughputController(145): 542548d310d66672741e6f9e560f852c#cf2#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:25,730 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/.tmp/cf1/e539327fe4594a8faf19441837ceb962 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/e539327fe4594a8faf19441837ceb962 2023-07-24 07:53:25,730 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/.tmp/cf2/071476384d4e4341ae62e07d2b730b57 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/071476384d4e4341ae62e07d2b730b57 2023-07-24 07:53:25,738 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 542548d310d66672741e6f9e560f852c/cf2 of 542548d310d66672741e6f9e560f852c into 071476384d4e4341ae62e07d2b730b57(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:25,738 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 542548d310d66672741e6f9e560f852c/cf1 of 542548d310d66672741e6f9e560f852c into e539327fe4594a8faf19441837ceb962(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:25,738 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 542548d310d66672741e6f9e560f852c: 2023-07-24 07:53:25,739 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 542548d310d66672741e6f9e560f852c: 2023-07-24 07:53:25,739 INFO [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c., storeName=542548d310d66672741e6f9e560f852c/cf2, priority=9, startTime=1690185205654; duration=0sec 2023-07-24 07:53:25,739 INFO [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c., storeName=542548d310d66672741e6f9e560f852c/cf1, priority=9, startTime=1690185205653; duration=0sec 2023-07-24 07:53:25,740 DEBUG [RS:2;jenkins-hbase4:37809-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:25,740 DEBUG [RS:2;jenkins-hbase4:37809-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:25,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37809] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:48864 deadline: 1690185265828, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5. is not online on jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:25,947 DEBUG [Listener at localhost/42537] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-07-24 07:53:25,966 INFO [Listener at localhost/42537] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-24 07:53:25,982 DEBUG [Listener at localhost/42537] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-07-24 07:53:26,024 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:26,037 INFO [Listener at localhost/42537] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-07-24 07:53:26,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-07-24 07:53:26,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:26,057 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185206056"}]},"ts":"1690185206056"} 2023-07-24 07:53:26,059 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-07-24 07:53:26,061 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-07-24 07:53:26,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5eca7056352c3c5c9619dde691b5b9b3, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=542548d310d66672741e6f9e560f852c, UNASSIGN}] 2023-07-24 07:53:26,064 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=542548d310d66672741e6f9e560f852c, UNASSIGN 2023-07-24 07:53:26,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-24 07:53:26,066 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=542548d310d66672741e6f9e560f852c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:26,066 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690185206066"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185206066"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185206066"}]},"ts":"1690185206066"} 2023-07-24 07:53:26,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 542548d310d66672741e6f9e560f852c, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:53:26,073 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5eca7056352c3c5c9619dde691b5b9b3, UNASSIGN 2023-07-24 07:53:26,074 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5eca7056352c3c5c9619dde691b5b9b3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:26,074 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690185206074"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185206074"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185206074"}]},"ts":"1690185206074"} 2023-07-24 07:53:26,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 5eca7056352c3c5c9619dde691b5b9b3, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:53:26,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-24 07:53:26,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:26,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5eca7056352c3c5c9619dde691b5b9b3, disabling compactions & flushes 2023-07-24 07:53:26,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. 2023-07-24 07:53:26,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. 2023-07-24 07:53:26,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. after waiting 0 ms 2023-07-24 07:53:26,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. 2023-07-24 07:53:26,228 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e-bottom] to archive 2023-07-24 07:53:26,230 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:26,238 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,240 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,243 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,246 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,278 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577-bottom] to archive 2023-07-24 07:53:26,280 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:26,283 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,286 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,287 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,289 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 07:53:26,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:26,300 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3. 2023-07-24 07:53:26,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5eca7056352c3c5c9619dde691b5b9b3: 2023-07-24 07:53:26,302 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:26,302 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 542548d310d66672741e6f9e560f852c 2023-07-24 07:53:26,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 542548d310d66672741e6f9e560f852c, disabling compactions & flushes 2023-07-24 07:53:26,302 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. 2023-07-24 07:53:26,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. 2023-07-24 07:53:26,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. after waiting 0 ms 2023-07-24 07:53:26,302 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. 2023-07-24 07:53:26,303 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5eca7056352c3c5c9619dde691b5b9b3, regionState=CLOSED 2023-07-24 07:53:26,304 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690185206303"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185206303"}]},"ts":"1690185206303"} 2023-07-24 07:53:26,306 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-9db03282f75a467a80913c9104b71cc3, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1365d727c5024f168b214733b173f0c3, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-c2eec162c33140bcb05144ddcc92623a] to archive 2023-07-24 07:53:26,308 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:26,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-07-24 07:53:26,314 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/1727ed0d75b54fc2974373066d59b8b9.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 5eca7056352c3c5c9619dde691b5b9b3, server=jenkins-hbase4.apache.org,37809,1690185199136 in 232 msec 2023-07-24 07:53:26,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5eca7056352c3c5c9619dde691b5b9b3, UNASSIGN in 252 msec 2023-07-24 07:53:26,317 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/fa761475e42e4086a390c3a5d9ace2fb.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,318 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/b571dc026b1d4fa29d46cc474574027a.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,320 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/3b09af552d514727a84ae414de420a8e.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,321 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-9db03282f75a467a80913c9104b71cc3 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-9db03282f75a467a80913c9104b71cc3 2023-07-24 07:53:26,323 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1365d727c5024f168b214733b173f0c3 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1365d727c5024f168b214733b173f0c3 2023-07-24 07:53:26,324 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-c2eec162c33140bcb05144ddcc92623a to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-c2eec162c33140bcb05144ddcc92623a 2023-07-24 07:53:26,327 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1301c11c17f74693ac13c654e3eb2939, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-6117bbd26e9e44f884a166f2fbff5bd1, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-a203a2c08df540e5816c328c5236de43] to archive 2023-07-24 07:53:26,328 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:26,330 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/01e8acff035c4f8099f478f93188752d.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,332 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/c0280b228cb94c9d94d02ba658f51355.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,333 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/220c737baeb7466fb9b74acc451fa4a1.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,335 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/0b9ea7e45d864b29bf075937fd6b2577.a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,336 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1301c11c17f74693ac13c654e3eb2939 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-1301c11c17f74693ac13c654e3eb2939 2023-07-24 07:53:26,338 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-6117bbd26e9e44f884a166f2fbff5bd1 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-6117bbd26e9e44f884a166f2fbff5bd1 2023-07-24 07:53:26,339 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-a203a2c08df540e5816c328c5236de43 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/testSplitTableRegionUnevenDaughter=a9f3298786bf73c446662e1bb906c6e5-a203a2c08df540e5816c328c5236de43 2023-07-24 07:53:26,347 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 07:53:26,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:26,350 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c. 2023-07-24 07:53:26,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 542548d310d66672741e6f9e560f852c: 2023-07-24 07:53:26,352 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 542548d310d66672741e6f9e560f852c 2023-07-24 07:53:26,353 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=542548d310d66672741e6f9e560f852c, regionState=CLOSED 2023-07-24 07:53:26,353 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690185206353"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185206353"}]},"ts":"1690185206353"} 2023-07-24 07:53:26,358 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-07-24 07:53:26,358 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 542548d310d66672741e6f9e560f852c, server=jenkins-hbase4.apache.org,37809,1690185199136 in 286 msec 2023-07-24 07:53:26,360 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-07-24 07:53:26,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=542548d310d66672741e6f9e560f852c, UNASSIGN in 296 msec 2023-07-24 07:53:26,362 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185206362"}]},"ts":"1690185206362"} 2023-07-24 07:53:26,364 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-07-24 07:53:26,365 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-07-24 07:53:26,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 320 msec 2023-07-24 07:53:26,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-24 07:53:26,373 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-07-24 07:53:26,379 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-07-24 07:53:26,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:26,388 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:26,391 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:26,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-24 07:53:26,400 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,400 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:26,400 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c 2023-07-24 07:53:26,406 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/recovered.edits] 2023-07-24 07:53:26,406 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/recovered.edits] 2023-07-24 07:53:26,406 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/recovered.edits] 2023-07-24 07:53:26,413 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1365d727c5024f168b214733b173f0c3 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1365d727c5024f168b214733b173f0c3 2023-07-24 07:53:26,413 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/e539327fe4594a8faf19441837ceb962 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf1/e539327fe4594a8faf19441837ceb962 2023-07-24 07:53:26,413 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/72c74ba0e1d14e9898e187f32269d07a to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf1/72c74ba0e1d14e9898e187f32269d07a 2023-07-24 07:53:26,415 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/1727ed0d75b54fc2974373066d59b8b9 2023-07-24 07:53:26,418 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/3b09af552d514727a84ae414de420a8e 2023-07-24 07:53:26,418 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/071476384d4e4341ae62e07d2b730b57 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/cf2/071476384d4e4341ae62e07d2b730b57 2023-07-24 07:53:26,419 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/6fb00563aba64b6e99d6ca5607e34866 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/cf2/6fb00563aba64b6e99d6ca5607e34866 2023-07-24 07:53:26,421 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/9db03282f75a467a80913c9104b71cc3 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/9db03282f75a467a80913c9104b71cc3 2023-07-24 07:53:26,424 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/b571dc026b1d4fa29d46cc474574027a 2023-07-24 07:53:26,425 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/recovered.edits/90.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c/recovered.edits/90.seqid 2023-07-24 07:53:26,426 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/542548d310d66672741e6f9e560f852c 2023-07-24 07:53:26,426 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/recovered.edits/90.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3/recovered.edits/90.seqid 2023-07-24 07:53:26,426 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/c2eec162c33140bcb05144ddcc92623a to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/c2eec162c33140bcb05144ddcc92623a 2023-07-24 07:53:26,427 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/5eca7056352c3c5c9619dde691b5b9b3 2023-07-24 07:53:26,428 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf1/fa761475e42e4086a390c3a5d9ace2fb 2023-07-24 07:53:26,431 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/01e8acff035c4f8099f478f93188752d 2023-07-24 07:53:26,433 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/0b9ea7e45d864b29bf075937fd6b2577 2023-07-24 07:53:26,435 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/1301c11c17f74693ac13c654e3eb2939 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/1301c11c17f74693ac13c654e3eb2939 2023-07-24 07:53:26,437 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/220c737baeb7466fb9b74acc451fa4a1 2023-07-24 07:53:26,439 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/6117bbd26e9e44f884a166f2fbff5bd1 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/6117bbd26e9e44f884a166f2fbff5bd1 2023-07-24 07:53:26,441 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/a203a2c08df540e5816c328c5236de43 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/a203a2c08df540e5816c328c5236de43 2023-07-24 07:53:26,443 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/cf2/c0280b228cb94c9d94d02ba658f51355 2023-07-24 07:53:26,447 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/recovered.edits/85.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5/recovered.edits/85.seqid 2023-07-24 07:53:26,448 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionUnevenDaughter/a9f3298786bf73c446662e1bb906c6e5 2023-07-24 07:53:26,448 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-07-24 07:53:26,452 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:26,476 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-07-24 07:53:26,483 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 07:53:26,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-24 07:53:26,537 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-07-24 07:53:26,539 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:26,539 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-07-24 07:53:26,539 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185206539"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:26,540 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185206539"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:26,540 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185206539"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:26,545 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 07:53:26,545 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a9f3298786bf73c446662e1bb906c6e5, NAME => 'testSplitTableRegionUnevenDaughter,,1690185202088.a9f3298786bf73c446662e1bb906c6e5.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 5eca7056352c3c5c9619dde691b5b9b3, NAME => 'testSplitTableRegionUnevenDaughter,,1690185204297.5eca7056352c3c5c9619dde691b5b9b3.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 542548d310d66672741e6f9e560f852c, NAME => 'testSplitTableRegionUnevenDaughter,26,1690185204297.542548d310d66672741e6f9e560f852c.', STARTKEY => '26', ENDKEY => ''}] 2023-07-24 07:53:26,545 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-07-24 07:53:26,545 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690185206545"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:26,549 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-07-24 07:53:26,553 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:26,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 173 msec 2023-07-24 07:53:26,566 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-24 07:53:26,567 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-24 07:53:26,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-24 07:53:26,700 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-07-24 07:53:26,718 INFO [Listener at localhost/42537] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=391 (was 359) Potentially hanging thread: hconnection-0x392b5020-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x392b5020-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:51896 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:49106 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:37809-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:37246 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:51824 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:36952 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:37240 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:51834 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:51956 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:51926 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:49150 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:49108 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:51800 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:49164 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:37262 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x392b5020-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:37286 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1219320831_17 at /127.0.0.1:36978 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=826 (was 694) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=350 (was 345) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=9039 (was 9130) 2023-07-24 07:53:26,733 INFO [Listener at localhost/42537] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=391, OpenFileDescriptor=826, MaxFileDescriptor=60000, SystemLoadAverage=350, ProcessCount=174, AvailableMemoryMB=9038 2023-07-24 07:53:26,733 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:26,739 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 07:53:26,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 07:53:26,742 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:26,744 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 07:53:26,745 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 07:53:26,748 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:26,749 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945 empty. 2023-07-24 07:53:26,750 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:26,750 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-07-24 07:53:26,762 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-07-24 07:53:26,764 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => 7005ae1188bcf4c865415273ba071945, NAME => 'testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp 2023-07-24 07:53:26,777 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:26,777 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing 7005ae1188bcf4c865415273ba071945, disabling compactions & flushes 2023-07-24 07:53:26,777 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:26,777 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:26,778 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. after waiting 0 ms 2023-07-24 07:53:26,778 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:26,778 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:26,778 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for 7005ae1188bcf4c865415273ba071945: 2023-07-24 07:53:26,780 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 07:53:26,782 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185206781"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185206781"}]},"ts":"1690185206781"} 2023-07-24 07:53:26,784 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 07:53:26,786 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 07:53:26,786 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185206786"}]},"ts":"1690185206786"} 2023-07-24 07:53:26,788 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-07-24 07:53:26,792 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:53:26,793 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:53:26,793 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:53:26,793 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:53:26,793 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:53:26,793 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, ASSIGN}] 2023-07-24 07:53:26,795 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, ASSIGN 2023-07-24 07:53:26,796 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:53:26,947 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:53:26,948 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=7005ae1188bcf4c865415273ba071945, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:26,948 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185206948"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185206948"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185206948"}]},"ts":"1690185206948"} 2023-07-24 07:53:26,951 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:27,104 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,104 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 07:53:27,108 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36410, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 07:53:27,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:27,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7005ae1188bcf4c865415273ba071945, NAME => 'testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:53:27,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:27,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:27,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:27,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:27,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:27,114 INFO [StoreOpener-7005ae1188bcf4c865415273ba071945-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:27,116 DEBUG [StoreOpener-7005ae1188bcf4c865415273ba071945-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1 2023-07-24 07:53:27,116 DEBUG [StoreOpener-7005ae1188bcf4c865415273ba071945-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1 2023-07-24 07:53:27,116 INFO [StoreOpener-7005ae1188bcf4c865415273ba071945-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7005ae1188bcf4c865415273ba071945 columnFamilyName cf1 2023-07-24 07:53:27,117 INFO [StoreOpener-7005ae1188bcf4c865415273ba071945-1] regionserver.HStore(310): Store=7005ae1188bcf4c865415273ba071945/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:27,117 INFO [StoreOpener-7005ae1188bcf4c865415273ba071945-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:27,118 DEBUG [StoreOpener-7005ae1188bcf4c865415273ba071945-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2 2023-07-24 07:53:27,119 DEBUG [StoreOpener-7005ae1188bcf4c865415273ba071945-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2 2023-07-24 07:53:27,120 INFO [StoreOpener-7005ae1188bcf4c865415273ba071945-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7005ae1188bcf4c865415273ba071945 columnFamilyName cf2 2023-07-24 07:53:27,121 INFO [StoreOpener-7005ae1188bcf4c865415273ba071945-1] regionserver.HStore(310): Store=7005ae1188bcf4c865415273ba071945/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:27,122 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:27,122 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:27,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:27,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:27,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:53:27,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7005ae1188bcf4c865415273ba071945; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62553687, jitterRate=-0.06787742674350739}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:27,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7005ae1188bcf4c865415273ba071945: 2023-07-24 07:53:27,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945., pid=27, masterSystemTime=1690185207104 2023-07-24 07:53:27,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:27,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:27,137 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=7005ae1188bcf4c865415273ba071945, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,138 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185207137"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185207137"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185207137"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185207137"}]},"ts":"1690185207137"} 2023-07-24 07:53:27,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-07-24 07:53:27,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 in 189 msec 2023-07-24 07:53:27,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-07-24 07:53:27,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, ASSIGN in 350 msec 2023-07-24 07:53:27,148 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 07:53:27,148 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185207148"}]},"ts":"1690185207148"} 2023-07-24 07:53:27,150 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-07-24 07:53:27,154 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 07:53:27,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 413 msec 2023-07-24 07:53:27,246 DEBUG [Listener at localhost/42537] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 07:53:27,251 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36420, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 07:53:27,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:27,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 07:53:27,263 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 07:53:27,263 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,263 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 07:53:27,263 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 07:53:27,264 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,264 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,265 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,265 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,266 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,265 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,266 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,266 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:27,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,267 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,267 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,268 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,268 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,268 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,268 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,268 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,268 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,268 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,268 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,271 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,272 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,272 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:27,272 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:27,272 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:27,272 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. started... 2023-07-24 07:53:27,273 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7005ae1188bcf4c865415273ba071945 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 07:53:27,273 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,273 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,274 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,274 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,274 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,274 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,285 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5 2023-07-24 07:53:27,306 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/79fceec19e9d4a6fac52f25269273464 2023-07-24 07:53:27,312 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5 2023-07-24 07:53:27,319 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:53:27,320 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/79fceec19e9d4a6fac52f25269273464 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464 2023-07-24 07:53:27,326 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:53:27,327 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7005ae1188bcf4c865415273ba071945 in 54ms, sequenceid=6, compaction requested=false 2023-07-24 07:53:27,327 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-07-24 07:53:27,328 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7005ae1188bcf4c865415273ba071945: 2023-07-24 07:53:27,328 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,328 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,330 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,330 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,330 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,330 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 07:53:27,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35db4c5[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:27,333 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 07:53:27,333 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,335 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,336 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,335 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,336 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,336 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:27,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,336 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,336 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,336 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,336 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:27,336 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,336 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,336 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:27,336 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,337 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,338 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,339 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,339 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,340 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,340 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,340 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:27,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 07:53:27,344 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 07:53:27,344 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:27,344 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:27,344 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 07:53:27,344 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:27,346 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,346 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,346 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,347 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,346 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,346 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,347 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,347 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,347 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,348 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,350 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,350 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,350 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,353 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,353 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,353 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,353 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,353 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,353 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,353 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,353 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,353 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,353 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 07:53:27,354 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:27,353 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,353 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,355 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:27,355 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,355 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,355 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:27,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,355 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,355 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,355 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,356 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,455 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:27,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:27,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:27,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:27,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 07:53:27,477 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 07:53:27,477 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,478 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 07:53:27,478 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 07:53:27,479 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,479 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,480 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,480 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,480 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,480 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,480 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,481 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:27,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,482 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,482 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,482 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,482 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:27,482 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,484 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:27,484 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,484 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,482 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:27,485 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,485 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,485 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,484 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,485 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,485 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,485 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,485 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,486 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,485 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,486 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:27,486 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,486 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:27,486 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:27,487 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. started... 2023-07-24 07:53:27,486 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,487 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,487 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7005ae1188bcf4c865415273ba071945 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:27,488 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,490 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,503 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/750d554a6be24590968f50070bb11caf 2023-07-24 07:53:27,521 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/ad12f2195565455c8a02f8bd9e6b59ae 2023-07-24 07:53:27,527 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/750d554a6be24590968f50070bb11caf as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf 2023-07-24 07:53:27,532 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:53:27,533 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/ad12f2195565455c8a02f8bd9e6b59ae as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae 2023-07-24 07:53:27,539 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:53:27,540 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7005ae1188bcf4c865415273ba071945 in 53ms, sequenceid=19, compaction requested=false 2023-07-24 07:53:27,541 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7005ae1188bcf4c865415273ba071945: 2023-07-24 07:53:27,541 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:27,541 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:27,541 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:27,541 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,541 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,541 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,543 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,543 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,543 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,543 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 07:53:27,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7eca93b6[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:27,545 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 07:53:27,546 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,547 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,547 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,547 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,547 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,547 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,547 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:27,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,547 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,547 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,547 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,547 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:27,547 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:27,547 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,547 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,548 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:27,552 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,552 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,552 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,552 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,552 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,552 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,553 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,553 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,554 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,554 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:27,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 07:53:27,560 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 07:53:27,560 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:27,560 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:27,560 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 07:53:27,560 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:27,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,563 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,563 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,563 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,564 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,565 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,568 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,568 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,568 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,572 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,572 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,572 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,572 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,573 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,572 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 07:53:27,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:27,573 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,573 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,572 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,572 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,573 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:27,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,573 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:27,573 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,574 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,574 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,574 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,574 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,574 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,574 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,575 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,674 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:27,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:27,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:27,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:27,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 07:53:27,695 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 07:53:27,695 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,696 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 07:53:27,696 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 07:53:27,696 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,696 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,697 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,697 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,697 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,697 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,698 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,698 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,698 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,699 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,699 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,699 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,699 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,699 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,699 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:27,700 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,700 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:27,700 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,700 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,700 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,700 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,701 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,702 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,702 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:27,702 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:27,702 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:27,702 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. started... 2023-07-24 07:53:27,703 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7005ae1188bcf4c865415273ba071945 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:27,704 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,705 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,705 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,705 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,705 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,705 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,718 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/a9772057404d482889caf78ccff4df54 2023-07-24 07:53:27,736 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/f458ff4cfaec4dc5bd1b969c45141c47 2023-07-24 07:53:27,741 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/a9772057404d482889caf78ccff4df54 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54 2023-07-24 07:53:27,747 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:53:27,748 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/f458ff4cfaec4dc5bd1b969c45141c47 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47 2023-07-24 07:53:27,753 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:53:27,755 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7005ae1188bcf4c865415273ba071945 in 53ms, sequenceid=32, compaction requested=true 2023-07-24 07:53:27,755 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7005ae1188bcf4c865415273ba071945: 2023-07-24 07:53:27,755 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:27,755 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:27,755 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:27,755 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,755 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,755 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,758 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,758 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,758 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,759 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 07:53:27,761 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b7bb77e[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:27,761 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 07:53:27,761 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,762 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,762 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,762 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,762 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,763 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:27,762 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,763 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,763 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,763 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:27,762 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,763 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,763 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,763 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:27,763 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:27,764 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,764 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,764 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,764 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,764 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,764 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,765 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,765 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,765 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,765 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:27,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 07:53:27,769 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 07:53:27,769 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:27,769 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:27,769 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 07:53:27,769 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:27,772 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,772 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,772 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,772 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,772 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,772 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,772 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,773 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,773 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,774 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,775 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,776 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,776 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,779 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,779 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,779 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,779 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,779 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,779 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,779 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,780 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 07:53:27,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:27,780 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,780 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,780 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,780 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,780 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,780 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,780 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,780 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:27,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,781 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:27,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,781 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,781 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,781 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,881 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:27,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:27,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:27,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:27,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 07:53:27,905 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 07:53:27,906 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,906 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 07:53:27,907 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 07:53:27,907 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,907 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,908 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,908 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,908 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,908 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,909 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,909 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:27,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,910 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,910 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,910 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:27,910 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,911 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:27,911 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,911 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:27,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:27,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,914 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:27,914 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,914 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:27,914 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,914 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:27,914 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:27,916 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,916 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,917 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,916 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:27,917 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,917 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,917 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:27,917 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:27,917 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,917 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:27,917 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. started... 2023-07-24 07:53:27,917 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7005ae1188bcf4c865415273ba071945 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:27,923 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,923 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,923 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,923 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,924 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,924 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,931 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/ff638c3245b94a1c8c2810e188b6499c 2023-07-24 07:53:27,950 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/bfef71e695e2484a8e12b1c1da7205ed 2023-07-24 07:53:27,956 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/ff638c3245b94a1c8c2810e188b6499c as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c 2023-07-24 07:53:27,962 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:53:27,963 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/bfef71e695e2484a8e12b1c1da7205ed as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed 2023-07-24 07:53:27,969 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:53:27,970 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7005ae1188bcf4c865415273ba071945 in 53ms, sequenceid=45, compaction requested=true 2023-07-24 07:53:27,970 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7005ae1188bcf4c865415273ba071945: 2023-07-24 07:53:27,970 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:27,971 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:27,971 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:27,971 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,971 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:27,971 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:27,973 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,973 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,973 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,973 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:27,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,975 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 07:53:27,975 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b2c419a[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:27,975 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 07:53:27,975 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,978 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,978 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,978 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,978 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,978 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,978 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:27,978 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,979 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,978 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:27,979 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,979 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:27,979 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:27,979 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:27,979 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:27,980 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,980 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,980 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,981 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,981 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,981 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:27,981 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,981 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,981 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:27,982 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:27,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:27,986 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 07:53:27,986 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 07:53:27,986 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:27,986 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:27,986 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 07:53:27,986 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:27,988 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,988 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,988 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,988 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,988 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,988 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,988 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:27,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:27,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:27,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:27,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:27,991 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:27,991 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:27,992 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:27,993 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:27,994 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,994 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,998 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:27,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:27,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:27,999 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,000 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,000 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,000 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,000 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,000 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:28,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 07:53:28,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:28,001 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:28,001 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:28,016 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 07:53:28,016 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-24 07:53:28,017 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:28,017 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-07-24 07:53:28,101 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:28,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:28,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:28,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:28,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 07:53:28,125 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 07:53:28,125 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:28,125 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 07:53:28,125 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 07:53:28,125 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,125 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,127 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,127 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,127 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,127 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,127 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,127 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:28,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,128 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,128 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,128 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,128 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:28,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,129 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:28,129 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:28,129 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:28,129 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:28,129 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:28,129 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:28,129 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:28,129 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:28,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:28,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,129 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:28,130 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,131 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,131 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:28,136 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:28,137 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:28,137 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:28,137 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:28,137 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:28,137 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:28,137 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:28,137 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:28,137 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:28,137 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:28,137 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:28,137 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:28,137 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:28,137 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:28,137 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:28,137 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. started... 2023-07-24 07:53:28,137 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7005ae1188bcf4c865415273ba071945 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:28,139 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,139 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,139 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:28,152 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/b3103134693643f1941ffebec9fb978f 2023-07-24 07:53:28,169 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/7ae83d425e6343ddaf124df05eade2de 2023-07-24 07:53:28,176 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/b3103134693643f1941ffebec9fb978f as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f 2023-07-24 07:53:28,181 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:53:28,182 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/7ae83d425e6343ddaf124df05eade2de as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de 2023-07-24 07:53:28,188 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:53:28,189 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7005ae1188bcf4c865415273ba071945 in 52ms, sequenceid=58, compaction requested=true 2023-07-24 07:53:28,189 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7005ae1188bcf4c865415273ba071945: 2023-07-24 07:53:28,189 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:28,189 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:28,189 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:28,189 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,189 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:28,189 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:28,192 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,192 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:28,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:28,193 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:28,193 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:28,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:28,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:28,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 07:53:28,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35b93da6[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:28,195 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 07:53:28,195 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,196 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,196 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,196 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,197 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,197 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:28,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,197 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:28,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,197 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:28,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,197 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:28,197 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:28,197 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:28,197 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:28,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,197 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:28,197 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:28,197 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:28,198 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:28,198 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:28,198 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:28,199 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,199 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:28,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:28,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:28,199 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:28,199 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:28,201 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:28,201 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:28,201 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:28,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:28,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 07:53:28,204 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 07:53:28,204 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:28,204 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:28,204 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 07:53:28,204 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:28,206 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,206 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,206 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,206 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,206 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,206 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,206 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:28,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:28,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:28,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,213 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,213 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,213 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,215 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,215 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,215 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,220 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,220 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,220 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,220 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,221 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:28,220 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 07:53:28,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:28,221 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,220 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,220 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,221 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,221 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,221 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,221 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:28,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,222 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:28,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,221 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,221 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,222 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,222 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,222 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,222 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,322 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:28,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:28,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:28,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:28,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 07:53:28,344 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 07:53:28,344 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:28,344 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 07:53:28,344 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 07:53:28,344 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,344 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,345 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,345 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,345 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,345 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,346 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,346 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:28,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,347 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 07:53:28,347 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:28,348 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:28,348 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:28,348 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 07:53:28,348 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:28,348 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:28,348 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:28,348 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. started... 2023-07-24 07:53:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:28,349 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7005ae1188bcf4c865415273ba071945 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:28,350 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,350 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,350 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,350 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,350 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:28,350 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:28,361 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/d79ed41c4a034af8907d7204e5575446 2023-07-24 07:53:28,382 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/0fa647cbc652400dac022c8efb752582 2023-07-24 07:53:28,392 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/d79ed41c4a034af8907d7204e5575446 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446 2023-07-24 07:53:28,398 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:53:28,400 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/0fa647cbc652400dac022c8efb752582 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582 2023-07-24 07:53:28,406 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:53:28,407 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7005ae1188bcf4c865415273ba071945 in 58ms, sequenceid=71, compaction requested=true 2023-07-24 07:53:28,407 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7005ae1188bcf4c865415273ba071945: 2023-07-24 07:53:28,407 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:28,407 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:28,408 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:28,408 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,408 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 07:53:28,408 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 07:53:28,410 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,410 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:28,411 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,411 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:28,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:28,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:28,413 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 07:53:28,413 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fc45f84[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:28,413 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 07:53:28,413 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,415 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,415 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,415 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:28,415 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,415 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:28,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 07:53:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 07:53:28,415 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 07:53:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:28,417 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,418 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:28,417 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 07:53:28,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:28,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:28,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:28,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:28,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:28,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:28,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 07:53:28,421 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 07:53:28,422 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:28,422 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:28,422 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 07:53:28,422 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:28,425 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,425 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,425 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:28,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,425 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:28,425 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,425 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,425 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,426 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:28,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,426 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,427 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:28,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,429 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:28,429 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 07:53:28,429 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:28,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,433 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 07:53:28,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 07:53:28,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:28,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:28,433 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:28,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,434 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,434 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:28,434 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:28,434 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:28,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,434 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:28,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:28,434 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 07:53:28,435 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 07:53:28,534 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:28,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:28,798 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-24 07:53:28,799 DEBUG [Listener at localhost/42537] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:28,801 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:28,802 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:28,802 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:28,802 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:28,803 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:28,803 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:28,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:28,803 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:29,051 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,052 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:29,052 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:29,052 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:29,054 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-24 07:53:29,055 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:29,055 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:29,055 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:29,055 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:29,056 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:29,059 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-24 07:53:29,060 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:29,061 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:29,061 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:29,061 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:29,061 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,062 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:29,062 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,063 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,063 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:29,063 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,064 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:29,064 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,064 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,064 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,064 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-24 07:53:29,065 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:29,066 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:29,067 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185239067 2023-07-24 07:53:29,068 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:29,068 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,072 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:29,072 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=7005ae1188bcf4c865415273ba071945, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:53:29,072 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:29,073 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185269073 2023-07-24 07:53:29,073 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185329073 2023-07-24 07:53:29,073 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-24 07:53:29,073 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:29,075 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:29,075 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:29,075 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:29,323 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,323 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:29,323 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:29,323 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:29,324 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:29,324 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:29,324 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:29,324 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:29,324 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:29,325 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:29,326 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-24 07:53:29,327 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:29,327 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:29,328 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:29,328 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:29,328 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,329 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:29,329 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,329 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,329 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:29,330 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,330 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:29,330 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,330 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,330 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,330 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 07:53:29,331 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:29,331 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:29,332 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185239332 2023-07-24 07:53:29,332 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:29,333 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,336 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:29,336 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=7005ae1188bcf4c865415273ba071945, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:53:29,336 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:29,336 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185269336 2023-07-24 07:53:29,336 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185329336 2023-07-24 07:53:29,336 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 07:53:29,336 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:29,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN}] 2023-07-24 07:53:29,339 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:29,339 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:29,339 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:29,586 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,587 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:29,587 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:29,587 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:29,587 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:29,587 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:29,587 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:29,587 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:29,587 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:29,588 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:29,589 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-24 07:53:29,590 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:29,590 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:29,590 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:29,590 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:29,591 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,591 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:29,592 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,592 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,592 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:29,592 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,593 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:29,593 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,593 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,593 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,593 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 07:53:29,594 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:29,594 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:29,594 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185239594 2023-07-24 07:53:29,596 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:29,596 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,598 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:29,599 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=7005ae1188bcf4c865415273ba071945, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:53:29,599 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:29,599 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185269599 2023-07-24 07:53:29,599 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185329599 2023-07-24 07:53:29,599 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 07:53:29,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN}] 2023-07-24 07:53:29,599 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:29,602 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:29,603 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7005ae1188bcf4c865415273ba071945, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:29,603 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185209602"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185209602"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185209602"}]},"ts":"1690185209602"} 2023-07-24 07:53:29,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:29,605 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:29,605 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:29,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:29,605 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:29,849 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,849 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:29,849 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:29,850 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:29,851 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:29,851 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:29,851 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:29,851 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:29,851 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:29,852 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:29,854 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 07:53:29,854 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 07:53:29,855 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:29,855 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:29,856 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:29,856 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:29,857 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,857 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:29,857 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,858 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:29,858 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:29,860 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,860 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:29,860 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:29,860 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,860 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,860 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:29,860 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:29,861 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:29,861 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-24 07:53:29,861 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:29,862 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:29,862 INFO [Listener at localhost/42537] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945 to restore RIT 2023-07-24 07:53:29,863 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185239863 2023-07-24 07:53:29,863 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:29,864 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:29,866 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:29,867 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=7005ae1188bcf4c865415273ba071945, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:53:29,867 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:29,867 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185269867 2023-07-24 07:53:29,867 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185329867 2023-07-24 07:53:29,867 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:53:29,867 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7005ae1188bcf4c865415273ba071945, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:29,868 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185209867"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185209867"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185209867"}]},"ts":"1690185209867"} 2023-07-24 07:53:29,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:29,870 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:30,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7005ae1188bcf4c865415273ba071945, disabling compactions & flushes 2023-07-24 07:53:30,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:30,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:30,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. after waiting 0 ms 2023-07-24 07:53:30,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:30,024 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 7005ae1188bcf4c865415273ba071945 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 07:53:30,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/723287be49c34c5098bd025ac69965b5 2023-07-24 07:53:30,051 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/263a68b36cfa4a1f9eb2fb8f12501866 2023-07-24 07:53:30,056 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf1/723287be49c34c5098bd025ac69965b5 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5 2023-07-24 07:53:30,062 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:53:30,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/.tmp/cf2/263a68b36cfa4a1f9eb2fb8f12501866 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866 2023-07-24 07:53:30,068 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:53:30,069 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7005ae1188bcf4c865415273ba071945 in 45ms, sequenceid=82, compaction requested=true 2023-07-24 07:53:30,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 07:53:30,090 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:30,091 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. 2023-07-24 07:53:30,091 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7005ae1188bcf4c865415273ba071945: 2023-07-24 07:53:30,093 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,094 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7005ae1188bcf4c865415273ba071945, regionState=CLOSED 2023-07-24 07:53:30,095 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185210094"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185210094"}]},"ts":"1690185210094"} 2023-07-24 07:53:30,096 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:30,096 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:30,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:30,097 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:30,117 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,117 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:30,118 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:30,118 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:30,118 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:30,119 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:30,119 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:30,119 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:30,119 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:30,120 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:30,121 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 07:53:30,121 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-24 07:53:30,121 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-07-24 07:53:30,121 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:30,122 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:30,122 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:30,122 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:30,122 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:30,123 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:30,123 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:30,123 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:30,123 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:30,124 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,124 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:30,124 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:30,124 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:30,124 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,125 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,125 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,125 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:30,125 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:30,125 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:30,125 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 07:53:30,125 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:30,126 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:30,126 INFO [Listener at localhost/42537] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945 to restore RIT 2023-07-24 07:53:30,126 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185240126 2023-07-24 07:53:30,127 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:30,127 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:30,129 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:30,130 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=7005ae1188bcf4c865415273ba071945, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:53:30,130 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:30,130 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185270130 2023-07-24 07:53:30,130 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185330130 2023-07-24 07:53:30,130 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 07:53:30,130 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=7005ae1188bcf4c865415273ba071945, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:30,130 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185210130"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185210130"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185210130"}]},"ts":"1690185210130"} 2023-07-24 07:53:30,132 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:30,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-07-24 07:53:30,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 in 262 msec 2023-07-24 07:53:30,135 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:30,135 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:30,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:30,135 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:30,380 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,380 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:30,380 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:30,380 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:30,381 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:30,381 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:30,381 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:30,381 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:30,381 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:30,382 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:30,383 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 07:53:30,383 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-24 07:53:30,383 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:30,384 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:30,384 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:30,385 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:30,385 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:30,385 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:30,385 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:30,386 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:30,386 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:30,386 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:30,387 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,387 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:30,387 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:30,387 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:30,387 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,387 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,387 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,387 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:30,387 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:30,387 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 07:53:30,388 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:30,389 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:30,389 INFO [Listener at localhost/42537] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945 to restore RIT 2023-07-24 07:53:30,389 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185240389 2023-07-24 07:53:30,389 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:30,390 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:30,393 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:30,393 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=7005ae1188bcf4c865415273ba071945, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:53:30,393 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:30,393 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185270393 2023-07-24 07:53:30,393 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185330393 2023-07-24 07:53:30,394 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 07:53:30,394 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:30,396 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:30,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-24 07:53:30,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN in 797 msec 2023-07-24 07:53:30,402 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:30,402 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:30,402 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:30,644 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,644 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:30,644 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:30,644 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:30,644 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:30,644 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:30,644 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:30,644 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:30,645 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:30,645 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:30,647 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 07:53:30,647 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:30,647 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:30,647 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:30,648 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:30,648 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:30,648 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:30,648 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:30,649 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:30,649 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:30,649 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:30,649 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:30,650 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,650 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:30,650 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:30,650 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:30,650 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:30,650 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:30,651 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 07:53:30,651 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:30,652 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:30,652 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185240652 2023-07-24 07:53:30,654 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:30,654 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,656 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:30,656 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=7005ae1188bcf4c865415273ba071945, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:53:30,656 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:30,656 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185270656 2023-07-24 07:53:30,656 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185330656 2023-07-24 07:53:30,657 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 07:53:30,657 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,657 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,657 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,662 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:30,674 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=7005ae1188bcf4c865415273ba071945, threads=14 2023-07-24 07:53:30,675 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,676 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,676 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,676 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,676 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,676 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,677 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,677 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,677 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,677 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,677 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,678 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,678 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,678 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,863 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,882 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,895 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,897 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,899 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,903 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,904 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,904 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,916 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,917 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,920 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,927 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,927 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,930 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:30,930 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 7005ae1188bcf4c865415273ba071945 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 07:53:30,976 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:30,976 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:30,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:30,976 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:31,157 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,157 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:31,157 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:31,157 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:31,157 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:31,158 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:31,158 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:31,158 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:31,158 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:31,159 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:31,160 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-24 07:53:31,160 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:31,161 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:31,161 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:31,162 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:31,162 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:31,163 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:31,163 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:31,164 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:31,164 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:31,165 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:31,165 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:31,166 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,167 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:31,167 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:31,167 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:31,167 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,167 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,167 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,167 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:31,168 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:31,168 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-24 07:53:31,168 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:31,169 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:31,170 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185241170 2023-07-24 07:53:31,171 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,171 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:31,174 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:31,174 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=7005ae1188bcf4c865415273ba071945, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:53:31,175 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:31,175 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185271175 2023-07-24 07:53:31,175 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185331175 2023-07-24 07:53:31,175 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:53:31,187 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=7005ae1188bcf4c865415273ba071945, threads=14 2023-07-24 07:53:31,188 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,188 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,189 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,192 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,192 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,192 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,192 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,193 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,193 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,193 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,193 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,193 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,194 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,200 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,407 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,412 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,420 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,429 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,435 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,451 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,453 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,454 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,454 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,463 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,463 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,467 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,479 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,481 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5 for region: 7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:31,481 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 7005ae1188bcf4c865415273ba071945 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 07:53:31,530 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:31,536 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:53:31,538 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:53:31,538 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:31,538 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,538 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:31,538 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:31,675 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,676 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:31,676 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:31,676 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:31,676 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:31,676 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:31,676 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:31,676 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:31,676 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:31,677 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:31,678 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-24 07:53:31,678 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:31,679 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:31,679 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:31,679 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:31,680 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:31,680 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:31,680 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:31,681 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:31,681 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:31,682 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:31,682 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:31,683 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,683 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:31,683 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:31,683 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:31,684 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,684 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,684 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,684 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:31,684 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:31,684 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-24 07:53:31,685 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:31,686 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:31,686 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185241686 2023-07-24 07:53:31,687 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:31,687 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,690 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:31,691 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=7005ae1188bcf4c865415273ba071945, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:53:31,691 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:31,691 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185271691 2023-07-24 07:53:31,691 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185331691 2023-07-24 07:53:31,691 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 07:53:31,695 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:31,697 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:31,697 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:31,697 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:31,803 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 07:53:31,942 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:31,942 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:31,942 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:31,942 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:31,942 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:31,942 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:31,942 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:31,942 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:31,942 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:31,943 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:31,944 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-24 07:53:31,945 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:31,945 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:31,945 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:31,946 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:31,946 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:31,946 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:31,947 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:31,947 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:31,947 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:31,948 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:31,948 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:31,949 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,949 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN 2023-07-24 07:53:31,949 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:31,949 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:31,949 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,949 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,950 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,950 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=7005ae1188bcf4c865415273ba071945, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:31,950 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 7005ae1188bcf4c865415273ba071945, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:31,950 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 07:53:31,951 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:31,951 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:31,952 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185241952 2023-07-24 07:53:31,953 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:31,953 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:31,955 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:31,956 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=7005ae1188bcf4c865415273ba071945, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:53:31,956 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:31,956 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185271956 2023-07-24 07:53:31,956 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185331956 2023-07-24 07:53:31,956 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 07:53:31,960 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185211959"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1690185211959"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1690185211959"}]},"ts":"1690185211959"} 2023-07-24 07:53:31,960 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185211959"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185211959"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185211959"}]},"ts":"1690185211959"} 2023-07-24 07:53:31,960 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185211959"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185211959"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185211959"}]},"ts":"1690185211959"} 2023-07-24 07:53:31,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=efa5009ac14fc821e717f05c40ca0ce6, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cee8e570880880b54e8e6fcbd087cb82, ASSIGN}] 2023-07-24 07:53:31,968 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cee8e570880880b54e8e6fcbd087cb82, ASSIGN 2023-07-24 07:53:31,969 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cee8e570880880b54e8e6fcbd087cb82, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:53:31,971 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=efa5009ac14fc821e717f05c40ca0ce6, ASSIGN 2023-07-24 07:53:31,972 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=efa5009ac14fc821e717f05c40ca0ce6, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:53:32,120 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 07:53:32,122 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=efa5009ac14fc821e717f05c40ca0ce6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:32,122 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185212122"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185212122"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185212122"}]},"ts":"1690185212122"} 2023-07-24 07:53:32,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure efa5009ac14fc821e717f05c40ca0ce6, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:32,128 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=cee8e570880880b54e8e6fcbd087cb82, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:32,128 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185212128"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185212128"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185212128"}]},"ts":"1690185212128"} 2023-07-24 07:53:32,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure cee8e570880880b54e8e6fcbd087cb82, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:32,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. 2023-07-24 07:53:32,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => efa5009ac14fc821e717f05c40ca0ce6, NAME => 'testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.', STARTKEY => '', ENDKEY => '41'} 2023-07-24 07:53:32,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:32,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:32,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:32,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:32,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:32,283 INFO [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:32,283 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1 2023-07-24 07:53:32,284 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1 2023-07-24 07:53:32,284 INFO [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region efa5009ac14fc821e717f05c40ca0ce6 columnFamilyName cf1 2023-07-24 07:53:32,301 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5-bottom 2023-07-24 07:53:32,307 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5-bottom 2023-07-24 07:53:32,312 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf-bottom 2023-07-24 07:53:32,317 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54-bottom 2023-07-24 07:53:32,323 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f-bottom 2023-07-24 07:53:32,329 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446-bottom 2023-07-24 07:53:32,335 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c-bottom 2023-07-24 07:53:32,335 INFO [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(310): Store=efa5009ac14fc821e717f05c40ca0ce6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:32,335 INFO [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:32,336 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2 2023-07-24 07:53:32,336 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2 2023-07-24 07:53:32,336 INFO [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region efa5009ac14fc821e717f05c40ca0ce6 columnFamilyName cf2 2023-07-24 07:53:32,352 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582-bottom 2023-07-24 07:53:32,356 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866-bottom 2023-07-24 07:53:32,361 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464-bottom 2023-07-24 07:53:32,367 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de-bottom 2023-07-24 07:53:32,371 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae-bottom 2023-07-24 07:53:32,376 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed-bottom 2023-07-24 07:53:32,383 DEBUG [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47-bottom 2023-07-24 07:53:32,383 INFO [StoreOpener-efa5009ac14fc821e717f05c40ca0ce6-1] regionserver.HStore(310): Store=efa5009ac14fc821e717f05c40ca0ce6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:32,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:32,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:32,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:32,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:32,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened efa5009ac14fc821e717f05c40ca0ce6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69463809, jitterRate=0.03509141504764557}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:32,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for efa5009ac14fc821e717f05c40ca0ce6: 2023-07-24 07:53:32,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6., pid=33, masterSystemTime=1690185212277 2023-07-24 07:53:32,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:32,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 07:53:32,393 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:32,393 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:32,393 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. 2023-07-24 07:53:32,393 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. 2023-07-24 07:53:32,393 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1912): efa5009ac14fc821e717f05c40ca0ce6/cf1 is initiating minor compaction (all files) 2023-07-24 07:53:32,393 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1912): efa5009ac14fc821e717f05c40ca0ce6/cf2 is initiating minor compaction (all files) 2023-07-24 07:53:32,394 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2259): Starting compaction of efa5009ac14fc821e717f05c40ca0ce6/cf2 in testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. 2023-07-24 07:53:32,394 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866-bottom] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/.tmp, totalSize=34.9 K 2023-07-24 07:53:32,395 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of efa5009ac14fc821e717f05c40ca0ce6/cf1 in testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. 2023-07-24 07:53:32,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. 2023-07-24 07:53:32,395 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185207252 2023-07-24 07:53:32,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. 2023-07-24 07:53:32,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. 2023-07-24 07:53:32,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cee8e570880880b54e8e6fcbd087cb82, NAME => 'testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.', STARTKEY => '41', ENDKEY => ''} 2023-07-24 07:53:32,395 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5-bottom] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/.tmp, totalSize=34.9 K 2023-07-24 07:53:32,396 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=efa5009ac14fc821e717f05c40ca0ce6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:32,396 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185207458 2023-07-24 07:53:32,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:32,396 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185212396"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185212396"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185212396"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185212396"}]},"ts":"1690185212396"} 2023-07-24 07:53:32,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:32,396 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185207252 2023-07-24 07:53:32,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:32,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:32,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:32,396 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185207676 2023-07-24 07:53:32,397 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185207458 2023-07-24 07:53:32,397 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185207883 2023-07-24 07:53:32,400 INFO [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:32,400 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185207676 2023-07-24 07:53:32,400 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185208104 2023-07-24 07:53:32,401 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185207883 2023-07-24 07:53:32,401 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185208324 2023-07-24 07:53:32,402 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1 2023-07-24 07:53:32,402 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1 2023-07-24 07:53:32,402 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185208537 2023-07-24 07:53:32,403 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185208104 2023-07-24 07:53:32,403 INFO [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cee8e570880880b54e8e6fcbd087cb82 columnFamilyName cf1 2023-07-24 07:53:32,403 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185208324 2023-07-24 07:53:32,404 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185208537 2023-07-24 07:53:32,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-07-24 07:53:32,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure efa5009ac14fc821e717f05c40ca0ce6, server=jenkins-hbase4.apache.org,46543,1690185199091 in 274 msec 2023-07-24 07:53:32,407 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=efa5009ac14fc821e717f05c40ca0ce6, ASSIGN in 441 msec 2023-07-24 07:53:32,431 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] throttle.PressureAwareThroughputController(145): efa5009ac14fc821e717f05c40ca0ce6#cf1#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:32,431 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] throttle.PressureAwareThroughputController(145): efa5009ac14fc821e717f05c40ca0ce6#cf2#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:32,437 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5-top 2023-07-24 07:53:32,448 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5-top 2023-07-24 07:53:32,453 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf-top 2023-07-24 07:53:32,463 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54-top 2023-07-24 07:53:32,469 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f-top 2023-07-24 07:53:32,471 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/.tmp/cf1/27ec303a03b94466b1b0124c6f383fcc as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/27ec303a03b94466b1b0124c6f383fcc 2023-07-24 07:53:32,472 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/.tmp/cf2/34360796e3c94742b59bc4f0e97fa668 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/34360796e3c94742b59bc4f0e97fa668 2023-07-24 07:53:32,475 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446-top 2023-07-24 07:53:32,478 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in efa5009ac14fc821e717f05c40ca0ce6/cf1 of efa5009ac14fc821e717f05c40ca0ce6 into 27ec303a03b94466b1b0124c6f383fcc(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:32,478 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for efa5009ac14fc821e717f05c40ca0ce6: 2023-07-24 07:53:32,478 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6., storeName=efa5009ac14fc821e717f05c40ca0ce6/cf1, priority=9, startTime=1690185212392; duration=0sec 2023-07-24 07:53:32,478 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:32,479 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in efa5009ac14fc821e717f05c40ca0ce6/cf2 of efa5009ac14fc821e717f05c40ca0ce6 into 34360796e3c94742b59bc4f0e97fa668(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:32,479 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for efa5009ac14fc821e717f05c40ca0ce6: 2023-07-24 07:53:32,479 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6., storeName=efa5009ac14fc821e717f05c40ca0ce6/cf2, priority=9, startTime=1690185212392; duration=0sec 2023-07-24 07:53:32,479 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:32,480 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c-top 2023-07-24 07:53:32,480 INFO [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(310): Store=cee8e570880880b54e8e6fcbd087cb82/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:32,481 INFO [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:32,482 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2 2023-07-24 07:53:32,482 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2 2023-07-24 07:53:32,482 INFO [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cee8e570880880b54e8e6fcbd087cb82 columnFamilyName cf2 2023-07-24 07:53:32,498 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582-top 2023-07-24 07:53:32,502 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866-top 2023-07-24 07:53:32,506 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464-top 2023-07-24 07:53:32,511 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de-top 2023-07-24 07:53:32,515 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae-top 2023-07-24 07:53:32,520 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed-top 2023-07-24 07:53:32,524 DEBUG [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47-top 2023-07-24 07:53:32,525 INFO [StoreOpener-cee8e570880880b54e8e6fcbd087cb82-1] regionserver.HStore(310): Store=cee8e570880880b54e8e6fcbd087cb82/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:32,525 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:32,527 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:32,529 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:32,530 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:32,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cee8e570880880b54e8e6fcbd087cb82; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62419976, jitterRate=-0.06986987590789795}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:32,531 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cee8e570880880b54e8e6fcbd087cb82: 2023-07-24 07:53:32,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82., pid=34, masterSystemTime=1690185212277 2023-07-24 07:53:32,532 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:32,533 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:32,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 07:53:32,533 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:32,533 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. 2023-07-24 07:53:32,533 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1912): cee8e570880880b54e8e6fcbd087cb82/cf1 is initiating minor compaction (all files) 2023-07-24 07:53:32,533 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. 2023-07-24 07:53:32,533 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1912): cee8e570880880b54e8e6fcbd087cb82/cf2 is initiating minor compaction (all files) 2023-07-24 07:53:32,533 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of cee8e570880880b54e8e6fcbd087cb82/cf1 in testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. 2023-07-24 07:53:32,534 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2259): Starting compaction of cee8e570880880b54e8e6fcbd087cb82/cf2 in testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. 2023-07-24 07:53:32,534 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866-top] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/.tmp, totalSize=34.9 K 2023-07-24 07:53:32,534 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5-top] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/.tmp, totalSize=34.9 K 2023-07-24 07:53:32,535 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690185207252 2023-07-24 07:53:32,535 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690185207252 2023-07-24 07:53:32,535 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. 2023-07-24 07:53:32,535 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690185207458 2023-07-24 07:53:32,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. 2023-07-24 07:53:32,535 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690185207458 2023-07-24 07:53:32,536 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690185207676 2023-07-24 07:53:32,536 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=cee8e570880880b54e8e6fcbd087cb82, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:32,536 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690185207676 2023-07-24 07:53:32,536 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185212536"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185212536"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185212536"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185212536"}]},"ts":"1690185212536"} 2023-07-24 07:53:32,537 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690185207883 2023-07-24 07:53:32,537 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690185207883 2023-07-24 07:53:32,537 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690185208104 2023-07-24 07:53:32,538 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690185208104 2023-07-24 07:53:32,538 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690185208324 2023-07-24 07:53:32,538 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690185208324 2023-07-24 07:53:32,538 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690185208537 2023-07-24 07:53:32,539 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690185208537 2023-07-24 07:53:32,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-07-24 07:53:32,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure cee8e570880880b54e8e6fcbd087cb82, server=jenkins-hbase4.apache.org,46543,1690185199091 in 408 msec 2023-07-24 07:53:32,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-07-24 07:53:32,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cee8e570880880b54e8e6fcbd087cb82, ASSIGN in 576 msec 2023-07-24 07:53:32,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 in 3.7450 sec 2023-07-24 07:53:32,559 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] throttle.PressureAwareThroughputController(145): cee8e570880880b54e8e6fcbd087cb82#cf2#compaction#34 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:32,560 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] throttle.PressureAwareThroughputController(145): cee8e570880880b54e8e6fcbd087cb82#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:32,586 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/.tmp/cf1/21148c5b63784e7ba725452834406e56 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/21148c5b63784e7ba725452834406e56 2023-07-24 07:53:32,587 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/.tmp/cf2/8aea1b98ee5c42d6b2ea75bae1b279fb as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/8aea1b98ee5c42d6b2ea75bae1b279fb 2023-07-24 07:53:32,592 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cee8e570880880b54e8e6fcbd087cb82/cf1 of cee8e570880880b54e8e6fcbd087cb82 into 21148c5b63784e7ba725452834406e56(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:32,592 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for cee8e570880880b54e8e6fcbd087cb82: 2023-07-24 07:53:32,592 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82., storeName=cee8e570880880b54e8e6fcbd087cb82/cf1, priority=9, startTime=1690185212532; duration=0sec 2023-07-24 07:53:32,592 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:32,594 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cee8e570880880b54e8e6fcbd087cb82/cf2 of cee8e570880880b54e8e6fcbd087cb82 into 8aea1b98ee5c42d6b2ea75bae1b279fb(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:32,594 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for cee8e570880880b54e8e6fcbd087cb82: 2023-07-24 07:53:32,594 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82., storeName=cee8e570880880b54e8e6fcbd087cb82/cf2, priority=9, startTime=1690185212533; duration=0sec 2023-07-24 07:53:32,594 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:32,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46543] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:36420 deadline: 1690185272707, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945. is not online on jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:32,820 DEBUG [Listener at localhost/42537] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 07:53:32,829 INFO [Listener at localhost/42537] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-24 07:53:32,831 DEBUG [Listener at localhost/42537] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 07:53:32,848 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:32,850 INFO [Listener at localhost/42537] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-07-24 07:53:32,850 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-07-24 07:53:32,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:32,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-24 07:53:32,855 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185212855"}]},"ts":"1690185212855"} 2023-07-24 07:53:32,857 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-07-24 07:53:32,858 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-07-24 07:53:32,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=efa5009ac14fc821e717f05c40ca0ce6, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cee8e570880880b54e8e6fcbd087cb82, UNASSIGN}] 2023-07-24 07:53:32,860 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cee8e570880880b54e8e6fcbd087cb82, UNASSIGN 2023-07-24 07:53:32,861 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=cee8e570880880b54e8e6fcbd087cb82, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:32,861 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185212861"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185212861"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185212861"}]},"ts":"1690185212861"} 2023-07-24 07:53:32,863 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure cee8e570880880b54e8e6fcbd087cb82, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:32,865 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=efa5009ac14fc821e717f05c40ca0ce6, UNASSIGN 2023-07-24 07:53:32,866 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=efa5009ac14fc821e717f05c40ca0ce6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:32,866 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185212866"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185212866"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185212866"}]},"ts":"1690185212866"} 2023-07-24 07:53:32,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure efa5009ac14fc821e717f05c40ca0ce6, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:32,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-24 07:53:33,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:33,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cee8e570880880b54e8e6fcbd087cb82, disabling compactions & flushes 2023-07-24 07:53:33,015 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. 2023-07-24 07:53:33,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. 2023-07-24 07:53:33,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. after waiting 0 ms 2023-07-24 07:53:33,015 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. 2023-07-24 07:53:33,027 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5-top] to archive 2023-07-24 07:53:33,028 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:33,030 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,032 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,033 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,034 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,036 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,037 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,038 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,040 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866-top] to archive 2023-07-24 07:53:33,041 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:33,043 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,045 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,046 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,048 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,049 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,051 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,052 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 07:53:33,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:33,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82. 2023-07-24 07:53:33,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cee8e570880880b54e8e6fcbd087cb82: 2023-07-24 07:53:33,059 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:33,059 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:33,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing efa5009ac14fc821e717f05c40ca0ce6, disabling compactions & flushes 2023-07-24 07:53:33,059 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. 2023-07-24 07:53:33,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. 2023-07-24 07:53:33,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. after waiting 0 ms 2023-07-24 07:53:33,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. 2023-07-24 07:53:33,060 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=cee8e570880880b54e8e6fcbd087cb82, regionState=CLOSED 2023-07-24 07:53:33,060 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185213059"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185213059"}]},"ts":"1690185213059"} 2023-07-24 07:53:33,061 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5-bottom] to archive 2023-07-24 07:53:33,062 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:33,064 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-07-24 07:53:33,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure cee8e570880880b54e8e6fcbd087cb82, server=jenkins-hbase4.apache.org,46543,1690185199091 in 200 msec 2023-07-24 07:53:33,066 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=cee8e570880880b54e8e6fcbd087cb82, UNASSIGN in 206 msec 2023-07-24 07:53:33,067 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,068 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,069 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,071 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,072 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,074 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,076 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866-bottom] to archive 2023-07-24 07:53:33,076 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:33,078 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,079 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,080 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,081 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,082 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,084 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,085 DEBUG [StoreCloser-testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 07:53:33,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:33,089 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6. 2023-07-24 07:53:33,089 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for efa5009ac14fc821e717f05c40ca0ce6: 2023-07-24 07:53:33,091 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:33,092 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=efa5009ac14fc821e717f05c40ca0ce6, regionState=CLOSED 2023-07-24 07:53:33,092 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185213092"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185213092"}]},"ts":"1690185213092"} 2023-07-24 07:53:33,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-07-24 07:53:33,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure efa5009ac14fc821e717f05c40ca0ce6, server=jenkins-hbase4.apache.org,46543,1690185199091 in 226 msec 2023-07-24 07:53:33,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-07-24 07:53:33,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=efa5009ac14fc821e717f05c40ca0ce6, UNASSIGN in 237 msec 2023-07-24 07:53:33,100 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185213100"}]},"ts":"1690185213100"} 2023-07-24 07:53:33,101 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-07-24 07:53:33,103 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-07-24 07:53:33,105 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 253 msec 2023-07-24 07:53:33,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-24 07:53:33,158 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-07-24 07:53:33,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-07-24 07:53:33,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:33,161 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:33,162 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:33,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-07-24 07:53:33,171 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,171 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:33,171 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:33,173 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/recovered.edits] 2023-07-24 07:53:33,174 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/recovered.edits] 2023-07-24 07:53:33,174 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/recovered.edits] 2023-07-24 07:53:33,178 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/27ec303a03b94466b1b0124c6f383fcc to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf1/27ec303a03b94466b1b0124c6f383fcc 2023-07-24 07:53:33,179 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/.regioninfo to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/.regioninfo 2023-07-24 07:53:33,179 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5 2023-07-24 07:53:33,181 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/723287be49c34c5098bd025ac69965b5 2023-07-24 07:53:33,181 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/34360796e3c94742b59bc4f0e97fa668 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/cf2/34360796e3c94742b59bc4f0e97fa668 2023-07-24 07:53:33,185 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/750d554a6be24590968f50070bb11caf 2023-07-24 07:53:33,185 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,186 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/a9772057404d482889caf78ccff4df54 2023-07-24 07:53:33,187 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,188 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/.regioninfo to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/.regioninfo 2023-07-24 07:53:33,189 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/b3103134693643f1941ffebec9fb978f 2023-07-24 07:53:33,189 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,190 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/d79ed41c4a034af8907d7204e5575446 2023-07-24 07:53:33,190 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,192 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf1/ff638c3245b94a1c8c2810e188b6499c 2023-07-24 07:53:33,192 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,193 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/0a0947d5c07e40eba7c8f7e2ede3b4d5.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,194 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,194 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/723287be49c34c5098bd025ac69965b5.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,196 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/0fa647cbc652400dac022c8efb752582 2023-07-24 07:53:33,196 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,196 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/750d554a6be24590968f50070bb11caf.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,198 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/263a68b36cfa4a1f9eb2fb8f12501866 2023-07-24 07:53:33,198 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/a9772057404d482889caf78ccff4df54.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,201 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/79fceec19e9d4a6fac52f25269273464 2023-07-24 07:53:33,201 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,202 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/b3103134693643f1941ffebec9fb978f.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,203 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/7ae83d425e6343ddaf124df05eade2de 2023-07-24 07:53:33,203 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,204 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/d79ed41c4a034af8907d7204e5575446.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,205 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/ad12f2195565455c8a02f8bd9e6b59ae 2023-07-24 07:53:33,206 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,207 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf1/ff638c3245b94a1c8c2810e188b6499c.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,207 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/bfef71e695e2484a8e12b1c1da7205ed 2023-07-24 07:53:33,208 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,210 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/cf2/f458ff4cfaec4dc5bd1b969c45141c47 2023-07-24 07:53:33,211 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,212 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/0fa647cbc652400dac022c8efb752582.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,213 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,214 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/263a68b36cfa4a1f9eb2fb8f12501866.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,214 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/recovered.edits/85.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945/recovered.edits/85.seqid 2023-07-24 07:53:33,214 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cee8e570880880b54e8e6fcbd087cb82/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,215 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,216 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/79fceec19e9d4a6fac52f25269273464.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,218 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/7ae83d425e6343ddaf124df05eade2de.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,218 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/21148c5b63784e7ba725452834406e56 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf1/21148c5b63784e7ba725452834406e56 2023-07-24 07:53:33,219 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/ad12f2195565455c8a02f8bd9e6b59ae.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,221 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/bfef71e695e2484a8e12b1c1da7205ed.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,221 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/8aea1b98ee5c42d6b2ea75bae1b279fb to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/cf2/8aea1b98ee5c42d6b2ea75bae1b279fb 2023-07-24 07:53:33,222 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/efa5009ac14fc821e717f05c40ca0ce6/cf2/f458ff4cfaec4dc5bd1b969c45141c47.7005ae1188bcf4c865415273ba071945 2023-07-24 07:53:33,225 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/recovered.edits/90.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82/recovered.edits/90.seqid 2023-07-24 07:53:33,226 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:33,227 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/recovered.edits/90.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6/recovered.edits/90.seqid 2023-07-24 07:53:33,227 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitWithoutPONR/efa5009ac14fc821e717f05c40ca0ce6 2023-07-24 07:53:33,227 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-07-24 07:53:33,230 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:33,233 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-07-24 07:53:33,235 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-07-24 07:53:33,236 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:33,236 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-07-24 07:53:33,237 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185213236"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:33,237 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185213236"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:33,237 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185213236"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:33,239 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 07:53:33,239 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7005ae1188bcf4c865415273ba071945, NAME => 'testSplitWithoutPONR,,1690185206741.7005ae1188bcf4c865415273ba071945.', STARTKEY => '', ENDKEY => ''}, {ENCODED => efa5009ac14fc821e717f05c40ca0ce6, NAME => 'testSplitWithoutPONR,,1690185208799.efa5009ac14fc821e717f05c40ca0ce6.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => cee8e570880880b54e8e6fcbd087cb82, NAME => 'testSplitWithoutPONR,41,1690185208799.cee8e570880880b54e8e6fcbd087cb82.', STARTKEY => '41', ENDKEY => ''}] 2023-07-24 07:53:33,239 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-07-24 07:53:33,239 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690185213239"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:33,240 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-07-24 07:53:33,242 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:33,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 83 msec 2023-07-24 07:53:33,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-07-24 07:53:33,264 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-07-24 07:53:33,287 INFO [Listener at localhost/42537] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=402 (was 391) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:51896 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:49106 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:37246 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:51824 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:36952 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:49306 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:37240 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:51956 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:51926 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:49150 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:46543-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:49108 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:37262 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:42150 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:37286 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x392b5020-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:36978 [Waiting for operation #31] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:42158 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=852 (was 826) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=330 (was 350), ProcessCount=174 (was 174), AvailableMemoryMB=9034 (was 9038) 2023-07-24 07:53:33,303 INFO [Listener at localhost/42537] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=402, OpenFileDescriptor=852, MaxFileDescriptor=60000, SystemLoadAverage=330, ProcessCount=174, AvailableMemoryMB=9033 2023-07-24 07:53:33,304 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:33,309 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 07:53:33,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 07:53:33,312 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:33,314 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 07:53:33,315 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 07:53:33,317 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:33,317 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b empty. 2023-07-24 07:53:33,318 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:33,318 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-07-24 07:53:33,330 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-07-24 07:53:33,332 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3d3705bd1d338a8a463a7f16f68aba1b, NAME => 'testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp 2023-07-24 07:53:33,342 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:33,343 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 3d3705bd1d338a8a463a7f16f68aba1b, disabling compactions & flushes 2023-07-24 07:53:33,343 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:33,343 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:33,343 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. after waiting 0 ms 2023-07-24 07:53:33,343 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:33,343 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:33,343 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:33,345 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 07:53:33,346 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185213346"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185213346"}]},"ts":"1690185213346"} 2023-07-24 07:53:33,347 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 07:53:33,348 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 07:53:33,348 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185213348"}]},"ts":"1690185213348"} 2023-07-24 07:53:33,350 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-07-24 07:53:33,353 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:53:33,354 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:53:33,354 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:53:33,354 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:53:33,354 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:53:33,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN}] 2023-07-24 07:53:33,356 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:33,357 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46877,1690185199045; forceNewPlan=false, retain=false 2023-07-24 07:53:33,507 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:53:33,508 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=3d3705bd1d338a8a463a7f16f68aba1b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,508 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185213508"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185213508"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185213508"}]},"ts":"1690185213508"} 2023-07-24 07:53:33,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:53:33,663 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,663 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 07:53:33,667 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60912, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 07:53:33,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:33,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3d3705bd1d338a8a463a7f16f68aba1b, NAME => 'testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:53:33,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:33,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:33,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:33,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:33,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:33,675 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:33,676 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1 2023-07-24 07:53:33,677 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1 2023-07-24 07:53:33,677 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3d3705bd1d338a8a463a7f16f68aba1b columnFamilyName cf1 2023-07-24 07:53:33,678 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(310): Store=3d3705bd1d338a8a463a7f16f68aba1b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:33,678 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:33,679 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2 2023-07-24 07:53:33,679 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2 2023-07-24 07:53:33,680 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3d3705bd1d338a8a463a7f16f68aba1b columnFamilyName cf2 2023-07-24 07:53:33,680 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(310): Store=3d3705bd1d338a8a463a7f16f68aba1b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:33,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:33,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:33,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:33,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:33,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:53:33,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3d3705bd1d338a8a463a7f16f68aba1b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68078768, jitterRate=0.014452695846557617}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:33,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:33,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b., pid=43, masterSystemTime=1690185213663 2023-07-24 07:53:33,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:33,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:33,694 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=3d3705bd1d338a8a463a7f16f68aba1b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,695 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185213694"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185213694"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185213694"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185213694"}]},"ts":"1690185213694"} 2023-07-24 07:53:33,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-07-24 07:53:33,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 in 186 msec 2023-07-24 07:53:33,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-07-24 07:53:33,702 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN in 346 msec 2023-07-24 07:53:33,702 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 07:53:33,703 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185213702"}]},"ts":"1690185213702"} 2023-07-24 07:53:33,704 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-07-24 07:53:33,711 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 07:53:33,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 400 msec 2023-07-24 07:53:33,815 DEBUG [Listener at localhost/42537] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 07:53:33,820 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60918, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 07:53:33,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:33,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 07:53:33,830 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 07:53:33,830 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:33,830 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 07:53:33,830 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:53:33,830 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,830 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:33,831 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:33,831 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:33,831 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:33,832 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:33,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:33,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:33,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:33,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:33,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:33,832 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,832 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:33,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:33,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:33,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:33,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,833 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:33,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:33,833 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:33,834 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:33,834 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:33,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:33,834 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:33,834 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:33,834 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:33,834 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:33,834 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:33,834 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:33,834 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:33,835 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:33,835 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:33,834 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:33,835 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:33,835 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:33,835 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:33,835 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:33,835 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:33,835 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:33,835 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. started... 2023-07-24 07:53:33,835 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:33,835 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:33,836 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:33,836 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d3705bd1d338a8a463a7f16f68aba1b 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 07:53:33,838 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,838 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,839 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:33,839 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,839 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,839 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:33,849 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/2374a8a7f6e04140a3e5bb86774e3b34 2023-07-24 07:53:33,864 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/e94ae2acbfc74575955d2f49aeb021fe 2023-07-24 07:53:33,869 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/2374a8a7f6e04140a3e5bb86774e3b34 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/2374a8a7f6e04140a3e5bb86774e3b34 2023-07-24 07:53:33,874 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/2374a8a7f6e04140a3e5bb86774e3b34, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:53:33,875 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/e94ae2acbfc74575955d2f49aeb021fe as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/e94ae2acbfc74575955d2f49aeb021fe 2023-07-24 07:53:33,879 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/e94ae2acbfc74575955d2f49aeb021fe, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:53:33,880 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 3d3705bd1d338a8a463a7f16f68aba1b in 44ms, sequenceid=6, compaction requested=false 2023-07-24 07:53:33,880 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-07-24 07:53:33,881 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:33,881 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:33,881 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:33,881 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:33,881 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,881 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:33,881 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:33,883 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,883 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:33,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:33,884 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,884 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:33,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:33,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:33,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:33,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:33,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:33,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:33,886 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 07:53:33,886 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4952cb45[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:33,886 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:53:33,886 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,887 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,887 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,887 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:33,888 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,888 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:33,888 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:33,889 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:33,889 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:33,890 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:33,890 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:33,890 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,890 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:33,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:33,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:33,890 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:33,890 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:33,891 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:33,891 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:33,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:33,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:33,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:33,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:33,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:33,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:33,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:33,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:33,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:53:33,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:53:33,895 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 07:53:33,895 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:33,895 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:33,895 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 07:53:33,895 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:33,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:33,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:33,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:33,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:33,898 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:33,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:33,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:33,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:33,899 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:33,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:33,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:33,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:33,899 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:33,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:33,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:33,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:33,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:33,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,901 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:33,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:33,901 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:33,902 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:33,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:33,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:33,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:33,905 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:33,905 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:33,905 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:33,905 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:33,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:33,905 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:33,905 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 07:53:33,905 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:33,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:33,905 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:33,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:33,905 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:33,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:33,905 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:33,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:33,906 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:33,906 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:33,906 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:33,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:33,906 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:33,906 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:33,907 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:33,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:33,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:33,907 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:33,907 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,907 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:33,907 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:33,907 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:33,907 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,007 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:34,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:34,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:34,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:34,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 07:53:34,028 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 07:53:34,028 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,028 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 07:53:34,028 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:53:34,028 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,028 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,030 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,030 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,030 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,030 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,030 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,030 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:34,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,031 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,032 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,032 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,032 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,032 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,032 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,032 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,032 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,032 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,032 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:34,033 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:34,033 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:34,033 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. started... 2023-07-24 07:53:34,033 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d3705bd1d338a8a463a7f16f68aba1b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:34,034 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,034 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,034 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,039 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,040 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,040 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,040 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,040 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,041 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,041 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,041 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,041 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,043 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,043 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,043 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,053 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/8e4a072eb3524db3bdcbb63b049ef808 2023-07-24 07:53:34,070 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/076e78f8f9a749e5ab1e1aa364f5df5d 2023-07-24 07:53:34,075 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/8e4a072eb3524db3bdcbb63b049ef808 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8e4a072eb3524db3bdcbb63b049ef808 2023-07-24 07:53:34,081 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8e4a072eb3524db3bdcbb63b049ef808, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:53:34,082 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/076e78f8f9a749e5ab1e1aa364f5df5d as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/076e78f8f9a749e5ab1e1aa364f5df5d 2023-07-24 07:53:34,087 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/076e78f8f9a749e5ab1e1aa364f5df5d, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:53:34,089 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3d3705bd1d338a8a463a7f16f68aba1b in 56ms, sequenceid=19, compaction requested=false 2023-07-24 07:53:34,089 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:34,089 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:34,089 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:34,089 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:34,089 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,089 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,089 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,091 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,091 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,091 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,091 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 07:53:34,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70cb6364[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:34,093 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:53:34,094 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,095 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,095 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,095 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,095 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,095 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,095 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,095 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:34,095 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,095 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,095 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:34,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,095 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,096 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:34,097 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,097 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,097 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,097 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,101 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,101 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,101 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,102 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,102 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,102 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:53:34,105 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:53:34,105 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 07:53:34,105 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:34,105 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:34,105 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 07:53:34,105 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:34,107 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,107 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,107 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,107 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,107 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,107 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,107 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,108 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,108 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,109 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,110 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,110 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,111 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,113 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,113 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,113 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,113 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,114 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 07:53:34,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:34,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,114 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,115 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:34,115 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,115 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,115 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:34,215 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:34,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:34,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:34,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:34,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 07:53:34,234 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 07:53:34,234 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,234 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 07:53:34,234 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:53:34,235 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,235 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,237 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,237 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,237 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,237 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,238 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,238 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,239 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,239 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,239 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,239 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,239 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,239 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,240 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,239 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,240 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,240 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,240 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,240 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,240 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,240 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,240 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,240 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,240 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:34,240 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:34,240 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:34,240 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. started... 2023-07-24 07:53:34,240 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d3705bd1d338a8a463a7f16f68aba1b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:34,241 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,242 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,242 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,242 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,243 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,243 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,255 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd 2023-07-24 07:53:34,273 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/45953d4fca6844a383912ca2af35d812 2023-07-24 07:53:34,279 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd 2023-07-24 07:53:34,283 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:53:34,284 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/45953d4fca6844a383912ca2af35d812 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/45953d4fca6844a383912ca2af35d812 2023-07-24 07:53:34,288 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/45953d4fca6844a383912ca2af35d812, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:53:34,289 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3d3705bd1d338a8a463a7f16f68aba1b in 49ms, sequenceid=32, compaction requested=true 2023-07-24 07:53:34,289 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:34,289 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:34,290 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:34,290 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:34,290 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,290 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,290 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,292 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,292 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,292 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,292 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 07:53:34,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ce334d6[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:34,294 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:53:34,294 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,296 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,297 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,296 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,297 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,297 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,297 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,297 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:34,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,297 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:34,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,297 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,297 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,297 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:34,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,297 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,298 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,298 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:34,299 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,299 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,299 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,299 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,300 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,300 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,300 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,300 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,300 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,300 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:53:34,303 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:53:34,303 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 07:53:34,303 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:34,303 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:34,303 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 07:53:34,303 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:34,305 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,305 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,305 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,305 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,305 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,305 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,305 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,306 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,307 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,307 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,309 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,309 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,310 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,317 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,317 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,317 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,317 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,317 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,317 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,317 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,317 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 07:53:34,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:34,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,318 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:34,317 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,318 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:34,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,318 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,318 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,318 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,318 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,318 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,319 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,319 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,319 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,319 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,418 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:34,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:34,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:34,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:34,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 07:53:34,438 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 07:53:34,438 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,438 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 07:53:34,438 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:53:34,438 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,438 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,440 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,440 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,440 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,440 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,440 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,440 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:34,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,441 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,441 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,441 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,442 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,442 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,442 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,442 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:34,442 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:34,442 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:34,442 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. started... 2023-07-24 07:53:34,443 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d3705bd1d338a8a463a7f16f68aba1b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:34,444 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,444 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,444 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,444 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,444 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,444 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,455 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/db0043a832c240969ee9338f6467d4e8 2023-07-24 07:53:34,470 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/8c1f3e287b2f49e2bb864bffb376cb77 2023-07-24 07:53:34,475 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/db0043a832c240969ee9338f6467d4e8 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/db0043a832c240969ee9338f6467d4e8 2023-07-24 07:53:34,479 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/db0043a832c240969ee9338f6467d4e8, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:53:34,480 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/8c1f3e287b2f49e2bb864bffb376cb77 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8c1f3e287b2f49e2bb864bffb376cb77 2023-07-24 07:53:34,484 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8c1f3e287b2f49e2bb864bffb376cb77, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:53:34,485 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3d3705bd1d338a8a463a7f16f68aba1b in 42ms, sequenceid=45, compaction requested=true 2023-07-24 07:53:34,485 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:34,485 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:34,485 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:34,486 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:34,486 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,486 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,486 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,487 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,487 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,488 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,488 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,490 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 07:53:34,490 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@316672b1[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:34,490 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:53:34,490 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,492 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,492 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,492 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,492 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,492 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:34,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,492 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,492 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,492 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,492 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,492 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:34,492 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,492 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:34,492 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,492 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:34,494 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,494 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,494 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,494 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,495 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,495 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,495 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,495 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,495 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,495 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:53:34,498 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:53:34,498 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 07:53:34,498 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:34,498 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:34,498 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 07:53:34,498 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:34,500 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,500 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,500 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,500 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,500 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,500 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,500 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,501 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,501 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,502 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,503 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,504 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,504 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,508 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,508 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,508 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,508 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,508 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,509 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 07:53:34,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:34,509 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,509 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,509 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,509 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,508 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,509 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:34,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,509 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,509 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:34,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,509 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,510 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,510 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,510 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,510 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,510 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,610 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:34,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:34,627 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:34,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:34,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 07:53:34,629 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 07:53:34,629 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,629 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 07:53:34,629 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:53:34,629 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,630 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,631 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,631 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,631 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,631 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,631 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,631 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:34,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,633 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,633 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,633 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,633 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,633 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,633 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,633 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,633 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,633 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,633 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,633 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,633 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,633 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,633 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,633 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,633 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,633 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,633 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,634 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,634 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,634 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,634 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:34,634 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:34,634 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:34,634 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. started... 2023-07-24 07:53:34,634 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d3705bd1d338a8a463a7f16f68aba1b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,635 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,636 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,636 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,636 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,644 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/f64401129d5c4e55b0b9a2cc1810da3f 2023-07-24 07:53:34,659 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/48a5f3335cb94368bd6f70431a5d7905 2023-07-24 07:53:34,664 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/f64401129d5c4e55b0b9a2cc1810da3f as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/f64401129d5c4e55b0b9a2cc1810da3f 2023-07-24 07:53:34,669 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/f64401129d5c4e55b0b9a2cc1810da3f, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:53:34,669 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/48a5f3335cb94368bd6f70431a5d7905 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/48a5f3335cb94368bd6f70431a5d7905 2023-07-24 07:53:34,673 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/48a5f3335cb94368bd6f70431a5d7905, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:53:34,674 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3d3705bd1d338a8a463a7f16f68aba1b in 40ms, sequenceid=58, compaction requested=true 2023-07-24 07:53:34,674 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:34,674 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:34,674 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:34,675 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:34,675 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,675 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,675 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,677 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,677 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,677 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,677 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 07:53:34,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e318f3b[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:34,679 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:53:34,679 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,680 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,681 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,681 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,681 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,681 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,681 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,681 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:34,681 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:34,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,681 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,681 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,681 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,681 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:34,681 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,681 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:34,683 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,683 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,683 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,687 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,687 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,687 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,688 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,688 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,688 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,688 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:53:34,692 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:53:34,692 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 07:53:34,692 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:34,692 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:34,692 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 07:53:34,692 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:34,693 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,693 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,693 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,693 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,694 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,694 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,694 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,694 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,695 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,695 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,698 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,698 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,699 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,701 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,701 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,701 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,701 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,701 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,702 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,701 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,701 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,702 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 07:53:34,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:34,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,702 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,702 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,702 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,702 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:34,703 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:34,702 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,703 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,703 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,703 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,703 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,703 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,803 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:34,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:34,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:34,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:34,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 07:53:34,824 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 07:53:34,824 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,824 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 07:53:34,824 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:53:34,824 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,824 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,827 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,827 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,827 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,827 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,827 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,827 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:34,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:34,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,829 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,829 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 07:53:34,829 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,829 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,829 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,829 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 07:53:34,829 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,829 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 07:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 07:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,830 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,830 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:53:34,829 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,830 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:34,830 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:34,830 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:34,830 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. started... 2023-07-24 07:53:34,830 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 3d3705bd1d338a8a463a7f16f68aba1b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:34,831 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,832 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,832 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,832 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,832 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,832 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,841 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/8689839085834b3786192c7de266a47c 2023-07-24 07:53:34,855 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/8b846816768149f79c18764b02685d99 2023-07-24 07:53:34,860 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/8689839085834b3786192c7de266a47c as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8689839085834b3786192c7de266a47c 2023-07-24 07:53:34,864 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8689839085834b3786192c7de266a47c, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:53:34,865 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/8b846816768149f79c18764b02685d99 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8b846816768149f79c18764b02685d99 2023-07-24 07:53:34,869 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8b846816768149f79c18764b02685d99, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:53:34,870 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 3d3705bd1d338a8a463a7f16f68aba1b in 40ms, sequenceid=71, compaction requested=true 2023-07-24 07:53:34,870 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:34,870 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:34,870 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:34,870 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:34,871 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,871 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 07:53:34,871 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 07:53:34,872 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,872 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,872 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,873 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:34,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 07:53:34,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45ca01da[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:34,875 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:53:34,875 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,877 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,877 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,877 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,878 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,878 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,878 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,877 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,878 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:34,878 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,878 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:34,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,878 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:34,878 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:53:34,878 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 07:53:34,878 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:34,879 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,879 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,879 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,880 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,880 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,880 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,880 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 07:53:34,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,880 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,880 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,881 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 07:53:34,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:53:34,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:53:34,884 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 07:53:34,884 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:34,884 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:34,884 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 07:53:34,884 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:34,886 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,886 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,886 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,886 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,886 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:34,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:34,886 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,886 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:34,886 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,887 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,887 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:34,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:34,889 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 07:53:34,889 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,890 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,896 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,896 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,896 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,896 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:34,896 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 07:53:34,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:34,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,896 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,896 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:34,896 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:34,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:34,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,897 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:34,897 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:34,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:34,897 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:34,897 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,898 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:34,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 07:53:34,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:34,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:34,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:34,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 07:53:34,898 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 07:53:34,998 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:34,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:35,266 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-24 07:53:35,267 DEBUG [Listener at localhost/42537] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:35,269 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,270 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,270 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,270 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,271 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:35,271 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,271 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:35,271 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:35,519 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,519 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:35,519 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:35,519 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:35,520 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-07-24 07:53:35,520 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:35,520 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:35,520 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:35,520 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:35,521 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:35,522 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-24 07:53:35,522 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:35,522 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:35,523 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:35,523 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:35,523 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:35,523 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:35,524 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:35,524 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:35,524 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:35,524 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:35,524 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:35,525 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:35,525 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:35,525 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,525 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:35,525 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,525 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,526 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,526 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 07:53:35,526 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:35,527 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:35,527 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185245527 2023-07-24 07:53:35,529 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,529 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:35,531 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:35,532 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:35,532 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:35,532 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185275532 2023-07-24 07:53:35,532 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185335532 2023-07-24 07:53:35,532 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:53:35,532 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:35,534 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:35,534 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:35,535 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:35,782 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,782 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:35,783 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:35,783 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:35,783 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:35,783 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:35,783 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:35,783 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:35,783 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:35,784 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:35,785 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-24 07:53:35,786 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:35,786 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:35,786 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:35,786 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:35,787 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:35,787 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:35,787 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:35,787 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:35,788 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:35,788 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:35,788 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:35,788 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:35,789 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:35,789 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,789 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:35,789 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,789 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,789 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,789 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 07:53:35,790 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:35,791 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:35,791 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185245791 2023-07-24 07:53:35,793 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,793 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:35,796 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:35,796 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:35,796 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:35,796 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185275796 2023-07-24 07:53:35,796 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185335796 2023-07-24 07:53:35,796 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:53:35,797 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:35,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN}] 2023-07-24 07:53:35,799 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:35,799 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:35,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:35,799 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:36,047 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,047 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:36,047 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:36,047 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:36,048 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:36,048 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:36,048 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:36,048 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:36,048 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:36,049 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:36,050 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-24 07:53:36,051 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:36,051 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:36,051 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:36,051 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:36,052 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,052 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:36,052 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,052 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,052 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,053 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:36,053 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,054 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,054 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:36,054 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,054 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:36,054 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,054 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,055 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,055 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 07:53:36,055 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:36,056 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:36,056 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185246056 2023-07-24 07:53:36,058 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,058 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:36,061 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:36,061 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:36,061 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:36,061 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185276061 2023-07-24 07:53:36,061 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185336061 2023-07-24 07:53:36,061 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:53:36,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN}] 2023-07-24 07:53:36,062 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:36,064 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:36,065 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:36,065 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185216065"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185216065"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185216065"}]},"ts":"1690185216065"} 2023-07-24 07:53:36,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:53:36,067 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:36,067 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:36,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:36,067 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:36,312 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,312 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:36,312 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:36,312 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:36,312 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:36,312 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:36,312 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:36,312 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:36,313 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:36,313 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:36,314 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 07:53:36,314 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 07:53:36,315 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:36,315 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:36,315 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:36,315 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:36,316 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,316 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:36,316 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,316 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,317 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,317 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:36,317 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,317 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,317 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:36,318 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,318 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:36,318 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:36,318 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,318 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,319 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,319 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:36,319 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:36,319 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 07:53:36,320 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:36,320 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:36,320 INFO [Listener at localhost/42537] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b to restore RIT 2023-07-24 07:53:36,320 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185246320 2023-07-24 07:53:36,321 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:36,321 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:36,324 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:36,324 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:36,324 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:36,324 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185276324 2023-07-24 07:53:36,324 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185336324 2023-07-24 07:53:36,324 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 07:53:36,325 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:36,325 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185216325"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185216325"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185216325"}]},"ts":"1690185216325"} 2023-07-24 07:53:36,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:53:36,327 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:36,480 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:36,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3d3705bd1d338a8a463a7f16f68aba1b, disabling compactions & flushes 2023-07-24 07:53:36,480 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:36,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:36,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. after waiting 0 ms 2023-07-24 07:53:36,480 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:36,480 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 3d3705bd1d338a8a463a7f16f68aba1b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 07:53:36,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/da60249f6ea042ec9696767cfe028258 2023-07-24 07:53:36,505 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/69f7c7b9072b41f8a76def5515e9b81c 2023-07-24 07:53:36,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/da60249f6ea042ec9696767cfe028258 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/da60249f6ea042ec9696767cfe028258 2023-07-24 07:53:36,515 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/da60249f6ea042ec9696767cfe028258, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:53:36,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/69f7c7b9072b41f8a76def5515e9b81c as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/69f7c7b9072b41f8a76def5515e9b81c 2023-07-24 07:53:36,521 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/69f7c7b9072b41f8a76def5515e9b81c, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:53:36,521 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 3d3705bd1d338a8a463a7f16f68aba1b in 41ms, sequenceid=82, compaction requested=true 2023-07-24 07:53:36,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 07:53:36,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:36,531 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:36,531 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:36,533 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:36,534 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED 2023-07-24 07:53:36,534 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185216534"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185216534"}]},"ts":"1690185216534"} 2023-07-24 07:53:36,535 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:36,535 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:36,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:36,535 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:36,575 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,575 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:36,575 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:36,575 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:36,575 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:36,575 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:36,575 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:36,575 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:36,575 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:36,576 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:36,577 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 07:53:36,577 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-24 07:53:36,577 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-07-24 07:53:36,577 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:36,578 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:36,578 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:36,578 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:36,579 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,579 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:36,579 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,579 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,579 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,580 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:36,580 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,580 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,580 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:36,581 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,581 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:36,581 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:36,581 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:36,581 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,581 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,581 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,581 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:36,581 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:36,582 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:36,582 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 07:53:36,582 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:36,583 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:36,583 INFO [Listener at localhost/42537] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b to restore RIT 2023-07-24 07:53:36,590 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185246590 2023-07-24 07:53:36,592 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:36,592 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:36,594 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:36,594 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:36,594 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:36,594 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185276594 2023-07-24 07:53:36,594 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185336594 2023-07-24 07:53:36,595 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 07:53:36,595 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:36,595 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185216595"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185216595"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185216595"}]},"ts":"1690185216595"} 2023-07-24 07:53:36,597 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:36,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-07-24 07:53:36,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 in 270 msec 2023-07-24 07:53:36,600 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:36,600 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:36,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:36,600 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:36,845 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,845 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:36,845 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:36,845 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:36,847 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:36,848 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:36,848 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:36,848 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:36,848 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:36,849 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:36,850 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 07:53:36,850 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-24 07:53:36,850 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:36,850 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:36,851 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:36,851 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:36,851 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:36,851 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,852 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:36,852 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,852 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:36,852 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,853 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:36,853 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,853 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:36,853 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:36,854 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,854 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:36,854 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:36,855 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:36,855 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,855 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,855 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,855 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:36,855 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:36,855 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 07:53:36,856 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:36,856 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:36,856 INFO [Listener at localhost/42537] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b to restore RIT 2023-07-24 07:53:36,857 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185246857 2023-07-24 07:53:36,857 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:36,858 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:36,860 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:36,860 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:36,860 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:36,860 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185276860 2023-07-24 07:53:36,860 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185336860 2023-07-24 07:53:36,860 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 07:53:36,860 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:36,862 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:36,865 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-07-24 07:53:36,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN in 800 msec 2023-07-24 07:53:36,868 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:36,868 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:36,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:36,869 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:37,111 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,111 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:37,111 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:37,111 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:37,111 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:37,111 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:37,111 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:37,111 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:37,111 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:37,112 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:37,113 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 07:53:37,113 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:37,113 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:37,114 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:37,114 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:37,114 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:37,114 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:37,115 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:37,115 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:37,115 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:37,115 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:37,116 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:37,116 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:37,116 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:37,116 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:37,116 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:37,117 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,117 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:37,117 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:37,117 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:37,117 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:37,117 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:37,117 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 07:53:37,118 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:37,119 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:37,122 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185247122 2023-07-24 07:53:37,125 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,125 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:37,127 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:37,128 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:37,128 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:37,128 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185277128 2023-07-24 07:53:37,128 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185337128 2023-07-24 07:53:37,128 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:53:37,129 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,129 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,129 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,134 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:37,145 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=3d3705bd1d338a8a463a7f16f68aba1b, threads=14 2023-07-24 07:53:37,147 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/076e78f8f9a749e5ab1e1aa364f5df5d for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,147 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/45953d4fca6844a383912ca2af35d812 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,147 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/48a5f3335cb94368bd6f70431a5d7905 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,147 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/69f7c7b9072b41f8a76def5515e9b81c for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,147 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8b846816768149f79c18764b02685d99 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,147 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8c1f3e287b2f49e2bb864bffb376cb77 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,148 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/e94ae2acbfc74575955d2f49aeb021fe for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,149 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,149 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/2374a8a7f6e04140a3e5bb86774e3b34 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,151 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8689839085834b3786192c7de266a47c for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,152 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8e4a072eb3524db3bdcbb63b049ef808 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,152 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/da60249f6ea042ec9696767cfe028258 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,152 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/db0043a832c240969ee9338f6467d4e8 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,152 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/f64401129d5c4e55b0b9a2cc1810da3f for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,359 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/45953d4fca6844a383912ca2af35d812 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,370 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/076e78f8f9a749e5ab1e1aa364f5df5d for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,394 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/69f7c7b9072b41f8a76def5515e9b81c for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,394 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/48a5f3335cb94368bd6f70431a5d7905 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,397 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8c1f3e287b2f49e2bb864bffb376cb77 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,411 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,415 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8689839085834b3786192c7de266a47c for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,428 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8b846816768149f79c18764b02685d99 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,431 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/f64401129d5c4e55b0b9a2cc1810da3f for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,432 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/e94ae2acbfc74575955d2f49aeb021fe for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,433 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/da60249f6ea042ec9696767cfe028258 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,439 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8e4a072eb3524db3bdcbb63b049ef808 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,439 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/db0043a832c240969ee9338f6467d4e8 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,442 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/2374a8a7f6e04140a3e5bb86774e3b34 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,442 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 3d3705bd1d338a8a463a7f16f68aba1b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 07:53:37,497 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:37,497 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,497 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:37,498 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:37,629 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,629 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:37,629 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:37,629 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:37,629 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:37,629 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:37,629 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:37,629 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:37,630 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:37,630 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:37,632 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-24 07:53:37,632 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:37,632 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:37,632 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:37,633 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:37,633 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:37,633 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:37,633 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:37,634 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:37,634 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:37,634 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:37,634 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:37,635 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:37,635 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:37,635 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:37,635 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:37,636 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,636 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:37,636 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:37,637 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:37,637 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,637 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,637 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,637 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:37,637 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:37,637 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 07:53:37,638 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:37,639 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:37,639 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185247639 2023-07-24 07:53:37,641 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,641 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:37,643 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:37,643 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:37,643 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:37,643 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185277643 2023-07-24 07:53:37,643 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185337643 2023-07-24 07:53:37,643 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 07:53:37,653 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=3d3705bd1d338a8a463a7f16f68aba1b, threads=14 2023-07-24 07:53:37,654 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/076e78f8f9a749e5ab1e1aa364f5df5d for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,654 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/45953d4fca6844a383912ca2af35d812 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,654 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/48a5f3335cb94368bd6f70431a5d7905 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,654 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/69f7c7b9072b41f8a76def5515e9b81c for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,654 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8b846816768149f79c18764b02685d99 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,654 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8c1f3e287b2f49e2bb864bffb376cb77 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,655 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/e94ae2acbfc74575955d2f49aeb021fe for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,655 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,655 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/2374a8a7f6e04140a3e5bb86774e3b34 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,655 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8689839085834b3786192c7de266a47c for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,656 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8e4a072eb3524db3bdcbb63b049ef808 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,658 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/da60249f6ea042ec9696767cfe028258 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,659 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/db0043a832c240969ee9338f6467d4e8 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,662 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/f64401129d5c4e55b0b9a2cc1810da3f for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,784 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 07:53:37,823 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8c1f3e287b2f49e2bb864bffb376cb77 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,838 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/e94ae2acbfc74575955d2f49aeb021fe for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,858 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/2374a8a7f6e04140a3e5bb86774e3b34 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,868 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,886 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8689839085834b3786192c7de266a47c for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,887 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8b846816768149f79c18764b02685d99 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,904 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/69f7c7b9072b41f8a76def5515e9b81c for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,905 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/45953d4fca6844a383912ca2af35d812 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,906 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/48a5f3335cb94368bd6f70431a5d7905 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,923 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8e4a072eb3524db3bdcbb63b049ef808 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,935 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/db0043a832c240969ee9338f6467d4e8 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,935 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/076e78f8f9a749e5ab1e1aa364f5df5d for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,936 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/f64401129d5c4e55b0b9a2cc1810da3f for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,944 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/da60249f6ea042ec9696767cfe028258 for region: 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:37,944 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 3d3705bd1d338a8a463a7f16f68aba1b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 07:53:37,983 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:37,988 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/5a709f00c41c4ec47f7254db00f24be6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:53:37,992 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/9ab9b0c2f983da01540f56de416e8725/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:53:37,992 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:37,992 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:37,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:37,992 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:38,143 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,144 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:38,144 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:38,144 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:38,144 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:38,144 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:38,144 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:38,144 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:38,144 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:38,145 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:38,146 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-24 07:53:38,146 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:38,146 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:38,147 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:38,147 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:38,148 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:38,148 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:38,148 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,149 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:38,149 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,149 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,150 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,150 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:38,150 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,151 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,151 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:38,152 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,152 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:38,152 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:38,155 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:38,155 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,155 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,156 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,156 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:38,156 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:38,156 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-24 07:53:38,157 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:38,157 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:38,160 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185248160 2023-07-24 07:53:38,162 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,162 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:38,165 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:38,166 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:38,166 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:38,166 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185278166 2023-07-24 07:53:38,166 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185338166 2023-07-24 07:53:38,166 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:53:38,170 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:38,175 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:38,175 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:38,175 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:38,417 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,417 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:38,417 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:38,417 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:38,417 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:38,417 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:38,417 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:38,418 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:38,418 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:38,418 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:38,419 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-24 07:53:38,419 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:38,420 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:38,420 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:38,420 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:38,420 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:38,421 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:38,421 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,421 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:38,422 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,422 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,422 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,423 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:38,423 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,423 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,424 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:38,424 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,424 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:38,424 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:38,425 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:38,425 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,425 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,425 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,425 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:38,425 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:38,425 DEBUG [Listener at localhost/42537] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,425 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 07:53:38,426 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:38,427 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:38,427 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185248427 2023-07-24 07:53:38,427 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:38,431 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,433 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:38,433 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:38,433 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:38,433 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185278433 2023-07-24 07:53:38,433 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185338433 2023-07-24 07:53:38,433 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 6 msec 2023-07-24 07:53:38,434 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:38,447 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:38,447 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 07:53:38,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:38,447 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:38,685 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,685 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:38,685 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:38,685 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:38,686 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:38,686 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:38,686 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:38,686 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:38,686 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:38,687 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:38,688 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-07-24 07:53:38,688 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:38,688 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:38,688 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:38,689 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:38,689 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:38,689 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:38,689 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,690 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:38,690 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,690 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,690 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,690 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:38,690 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,691 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,691 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:38,691 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,691 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:38,692 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:38,692 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:38,692 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,692 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,692 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,692 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:38,692 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:38,692 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 07:53:38,693 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:38,693 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:38,693 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185248693 2023-07-24 07:53:38,696 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:38,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,699 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:38,699 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:38,699 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:38,700 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185278700 2023-07-24 07:53:38,700 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185338700 2023-07-24 07:53:38,700 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:53:38,700 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:38,703 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:38,703 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 07:53:38,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:38,703 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:38,951 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,951 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:38,951 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:38,951 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:38,951 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:38,951 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:38,951 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:38,951 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:38,952 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:38,952 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:38,953 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-24 07:53:38,953 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:38,953 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:38,953 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:38,954 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:38,954 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:38,954 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:38,954 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,954 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:38,955 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,955 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:38,955 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,955 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:38,955 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,955 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:38,956 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:38,956 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,956 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:38,956 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:38,957 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:38,957 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,957 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,957 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,957 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:38,957 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:38,957 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 07:53:38,958 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:38,958 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:38,959 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185248959 2023-07-24 07:53:38,960 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:38,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:38,963 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:38,963 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:38,963 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:38,963 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185278963 2023-07-24 07:53:38,963 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185338963 2023-07-24 07:53:38,963 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:53:38,963 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:38,966 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:38,966 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 07:53:38,966 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:38,966 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:39,213 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,214 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:39,214 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:39,214 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:39,214 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:39,214 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:39,214 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:39,214 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:39,214 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:39,215 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:39,216 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-24 07:53:39,216 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:39,216 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:39,216 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:39,216 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:39,217 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:39,217 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:39,217 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:39,217 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:39,217 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:39,217 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:39,218 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:39,218 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:39,218 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:39,218 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:39,218 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:39,219 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,219 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:39,219 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:39,219 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:39,219 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,219 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,219 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,219 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:39,219 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:39,220 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 07:53:39,220 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:39,221 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:39,221 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185249221 2023-07-24 07:53:39,221 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:39,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,225 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:39,225 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:39,225 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:39,226 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185279226 2023-07-24 07:53:39,226 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185339226 2023-07-24 07:53:39,226 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 07:53:39,226 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:39,230 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN] 2023-07-24 07:53:39,230 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:39,230 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 07:53:39,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:39,230 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:39,476 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,476 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:39,476 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:39,476 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:39,476 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:39,477 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:39,477 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:39,477 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:39,477 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:39,478 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:39,478 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-24 07:53:39,478 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:53:39,478 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:53:39,479 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 07:53:39,479 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:39,479 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:39,479 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:39,479 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:39,479 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:39,480 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:39,480 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:39,480 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:39,480 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:39,480 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:39,480 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:39,481 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:39,481 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:39,482 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,482 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:39,482 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:39,482 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:39,482 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:39,482 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,482 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,482 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,482 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:39,482 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:39,482 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:39,482 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 07:53:39,483 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:39,484 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:39,484 INFO [Listener at localhost/42537] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b to restore RIT 2023-07-24 07:53:39,484 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185249484 2023-07-24 07:53:39,485 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:39,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,487 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:39,489 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:39,489 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:39,489 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:39,490 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185279490 2023-07-24 07:53:39,490 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185339490 2023-07-24 07:53:39,490 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-24 07:53:39,490 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:39,495 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:39,496 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 07:53:39,496 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:39,496 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:39,740 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,740 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:39,740 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:39,740 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:39,741 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:39,742 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:39,742 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:39,742 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:39,742 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:39,743 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:39,743 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 07:53:39,743 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 07:53:39,744 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:39,744 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:39,744 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:39,744 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:39,744 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:39,745 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:39,745 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:39,745 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:39,745 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:39,745 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:39,745 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:39,746 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:39,746 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:39,746 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,746 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:39,746 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:39,746 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,747 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,747 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,747 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:39,747 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-07-24 07:53:39,747 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:39,748 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:39,748 INFO [Listener at localhost/42537] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b to restore RIT 2023-07-24 07:53:39,748 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185249748 2023-07-24 07:53:39,748 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:39,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:39,752 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:39,754 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:39,754 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:39,754 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:39,754 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185279754 2023-07-24 07:53:39,754 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185339754 2023-07-24 07:53:39,754 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-24 07:53:39,754 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:39,756 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:39,757 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 07:53:39,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:39,757 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:40,004 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,005 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:40,005 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:40,005 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:40,005 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:40,005 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:40,005 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:40,005 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:40,005 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:40,006 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:40,007 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-24 07:53:40,007 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 07:53:40,007 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:40,007 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:40,007 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:40,007 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:40,008 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:40,008 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:40,008 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:40,008 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:40,009 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:40,009 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:40,009 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:40,009 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:40,009 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:40,010 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,010 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:40,010 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:40,010 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,010 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,010 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,010 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:40,010 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 07:53:40,011 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:40,011 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:40,011 INFO [Listener at localhost/42537] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b to restore RIT 2023-07-24 07:53:40,012 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185250012 2023-07-24 07:53:40,014 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,014 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:40,015 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:40,017 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:40,017 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:40,017 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:40,017 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185280017 2023-07-24 07:53:40,017 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185340017 2023-07-24 07:53:40,018 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:53:40,018 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:40,020 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:40,020 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 07:53:40,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:40,020 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:40,268 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,268 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:40,268 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:40,268 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:40,268 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:40,268 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:40,268 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:40,268 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:40,268 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:40,269 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:40,270 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-24 07:53:40,270 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 07:53:40,270 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:40,271 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:40,271 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:40,271 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:40,271 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:40,271 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:40,271 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:40,272 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:40,272 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:40,272 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:40,272 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:40,272 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:40,272 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:40,273 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,273 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:40,273 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:40,273 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,273 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,273 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,273 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:53:40,273 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-07-24 07:53:40,274 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:40,274 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:40,275 INFO [Listener at localhost/42537] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b to restore RIT 2023-07-24 07:53:40,275 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185250275 2023-07-24 07:53:40,276 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:40,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,280 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:40,281 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:40,281 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:40,282 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:40,282 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185280282 2023-07-24 07:53:40,282 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185340282 2023-07-24 07:53:40,282 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-24 07:53:40,282 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:53:40,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 exec-time=5.0140 sec 2023-07-24 07:53:40,284 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:40,285 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,46877,1690185199045; forceNewPlan=false, retain=true 2023-07-24 07:53:40,435 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:53:40,437 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=3d3705bd1d338a8a463a7f16f68aba1b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:40,437 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185220437"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185220437"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185220437"}]},"ts":"1690185220437"} 2023-07-24 07:53:40,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:53:40,439 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:53:40,439 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:40,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:53:40,439 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:53:40,532 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:40,532 WARN [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-07-24 07:53:40,532 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:53:40,532 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:53:40,533 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:53:40,534 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:53:40,534 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:53:40,534 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:53:40,534 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:53:40,534 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:53:40,535 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:53:40,536 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-07-24 07:53:40,536 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:53:40,537 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:53:40,537 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:53:40,537 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:53:40,537 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:40,537 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:53:40,537 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:40,538 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:53:40,538 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:40,538 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:53:40,538 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:40,538 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:53:40,538 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:40,539 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:53:40,539 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:40,539 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:53:40,539 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:40,539 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-07-24 07:53:40,540 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:53:40,540 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:53:40,540 INFO [Listener at localhost/42537] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b to restore RIT 2023-07-24 07:53:40,540 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185250540 2023-07-24 07:53:40,541 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:53:40,541 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:53:40,544 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:53:40,544 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=3d3705bd1d338a8a463a7f16f68aba1b, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,46877,1690185199045, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045, openSeqNum=2 2023-07-24 07:53:40,544 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:53:40,544 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185280544 2023-07-24 07:53:40,544 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185340544 2023-07-24 07:53:40,544 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 07:53:40,544 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=3d3705bd1d338a8a463a7f16f68aba1b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:40,544 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185220544"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185220544"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185220544"}]},"ts":"1690185220544"} 2023-07-24 07:53:40,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:53:40,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:40,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3d3705bd1d338a8a463a7f16f68aba1b, NAME => 'testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:53:40,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:40,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:40,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:40,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:40,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:40,704 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:40,705 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1 2023-07-24 07:53:40,705 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1 2023-07-24 07:53:40,706 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3d3705bd1d338a8a463a7f16f68aba1b columnFamilyName cf1 2023-07-24 07:53:40,713 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd 2023-07-24 07:53:40,716 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/2374a8a7f6e04140a3e5bb86774e3b34 2023-07-24 07:53:40,720 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8689839085834b3786192c7de266a47c 2023-07-24 07:53:40,724 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8e4a072eb3524db3bdcbb63b049ef808 2023-07-24 07:53:40,728 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/da60249f6ea042ec9696767cfe028258 2023-07-24 07:53:40,732 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/db0043a832c240969ee9338f6467d4e8 2023-07-24 07:53:40,736 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/f64401129d5c4e55b0b9a2cc1810da3f 2023-07-24 07:53:40,736 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(310): Store=3d3705bd1d338a8a463a7f16f68aba1b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:40,736 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:40,737 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2 2023-07-24 07:53:40,737 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2 2023-07-24 07:53:40,738 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3d3705bd1d338a8a463a7f16f68aba1b columnFamilyName cf2 2023-07-24 07:53:40,745 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/076e78f8f9a749e5ab1e1aa364f5df5d 2023-07-24 07:53:40,748 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/45953d4fca6844a383912ca2af35d812 2023-07-24 07:53:40,752 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/48a5f3335cb94368bd6f70431a5d7905 2023-07-24 07:53:40,755 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/69f7c7b9072b41f8a76def5515e9b81c 2023-07-24 07:53:40,758 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8b846816768149f79c18764b02685d99 2023-07-24 07:53:40,761 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8c1f3e287b2f49e2bb864bffb376cb77 2023-07-24 07:53:40,765 DEBUG [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/e94ae2acbfc74575955d2f49aeb021fe 2023-07-24 07:53:40,765 INFO [StoreOpener-3d3705bd1d338a8a463a7f16f68aba1b-1] regionserver.HStore(310): Store=3d3705bd1d338a8a463a7f16f68aba1b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:40,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:40,767 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:40,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.splits 2023-07-24 07:53:40,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:40,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:40,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3d3705bd1d338a8a463a7f16f68aba1b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63194402, jitterRate=-0.058330029249191284}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:40,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:40,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b., pid=48, masterSystemTime=1690185220698 2023-07-24 07:53:40,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:40,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 07:53:40,774 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:40,774 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:40,775 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-24 07:53:40,775 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-24 07:53:40,776 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HStore(1912): 3d3705bd1d338a8a463a7f16f68aba1b/cf1 is initiating minor compaction (all files) 2023-07-24 07:53:40,776 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HStore(1912): 3d3705bd1d338a8a463a7f16f68aba1b/cf2 is initiating minor compaction (all files) 2023-07-24 07:53:40,776 INFO [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 3d3705bd1d338a8a463a7f16f68aba1b/cf1 in testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:40,776 INFO [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 3d3705bd1d338a8a463a7f16f68aba1b/cf2 in testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:40,777 INFO [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/2374a8a7f6e04140a3e5bb86774e3b34, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8e4a072eb3524db3bdcbb63b049ef808, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/db0043a832c240969ee9338f6467d4e8, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/f64401129d5c4e55b0b9a2cc1810da3f, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8689839085834b3786192c7de266a47c, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/da60249f6ea042ec9696767cfe028258] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp, totalSize=34.9 K 2023-07-24 07:53:40,777 INFO [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/e94ae2acbfc74575955d2f49aeb021fe, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/076e78f8f9a749e5ab1e1aa364f5df5d, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/45953d4fca6844a383912ca2af35d812, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8c1f3e287b2f49e2bb864bffb376cb77, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/48a5f3335cb94368bd6f70431a5d7905, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8b846816768149f79c18764b02685d99, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/69f7c7b9072b41f8a76def5515e9b81c] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp, totalSize=34.9 K 2023-07-24 07:53:40,777 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:40,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:40,777 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting 2374a8a7f6e04140a3e5bb86774e3b34, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185213820 2023-07-24 07:53:40,777 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting e94ae2acbfc74575955d2f49aeb021fe, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185213820 2023-07-24 07:53:40,777 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=3d3705bd1d338a8a463a7f16f68aba1b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:40,778 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting 8e4a072eb3524db3bdcbb63b049ef808, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185214009 2023-07-24 07:53:40,778 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185220777"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185220777"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185220777"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185220777"}]},"ts":"1690185220777"} 2023-07-24 07:53:40,778 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting 076e78f8f9a749e5ab1e1aa364f5df5d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185214009 2023-07-24 07:53:40,778 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting 1f2bfd8d5a3f425481a64f68c4a5aedd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185214217 2023-07-24 07:53:40,778 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting 45953d4fca6844a383912ca2af35d812, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185214217 2023-07-24 07:53:40,778 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting db0043a832c240969ee9338f6467d4e8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185214420 2023-07-24 07:53:40,779 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting 8c1f3e287b2f49e2bb864bffb376cb77, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185214420 2023-07-24 07:53:40,779 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting f64401129d5c4e55b0b9a2cc1810da3f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185214611 2023-07-24 07:53:40,779 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting 48a5f3335cb94368bd6f70431a5d7905, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185214611 2023-07-24 07:53:40,779 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting 8689839085834b3786192c7de266a47c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185214805 2023-07-24 07:53:40,779 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting 8b846816768149f79c18764b02685d99, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185214805 2023-07-24 07:53:40,780 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting da60249f6ea042ec9696767cfe028258, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185215000 2023-07-24 07:53:40,780 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting 69f7c7b9072b41f8a76def5515e9b81c, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185215000 2023-07-24 07:53:40,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-07-24 07:53:40,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 in 233 msec 2023-07-24 07:53:40,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN in 1.5540 sec 2023-07-24 07:53:40,795 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-07-24 07:53:40,801 DEBUG [Listener at localhost/42537] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-07-24 07:53:40,803 INFO [Listener at localhost/42537] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-07-24 07:53:40,807 INFO [RS:0;jenkins-hbase4:46877-shortCompactions-0] throttle.PressureAwareThroughputController(145): 3d3705bd1d338a8a463a7f16f68aba1b#cf1#compaction#50 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:40,808 INFO [RS:0;jenkins-hbase4:46877-longCompactions-0] throttle.PressureAwareThroughputController(145): 3d3705bd1d338a8a463a7f16f68aba1b#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:40,809 INFO [Listener at localhost/42537] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-07-24 07:53:40,809 INFO [Listener at localhost/42537] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-07-24 07:53:40,809 INFO [Listener at localhost/42537] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-07-24 07:53:40,835 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf2/088f999024ec49dd851587c82e8d49d1 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/088f999024ec49dd851587c82e8d49d1 2023-07-24 07:53:40,836 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/.tmp/cf1/8df97f45d2af45c39cd8abd1c8c19693 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8df97f45d2af45c39cd8abd1c8c19693 2023-07-24 07:53:40,844 INFO [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3d3705bd1d338a8a463a7f16f68aba1b/cf1 of 3d3705bd1d338a8a463a7f16f68aba1b into 8df97f45d2af45c39cd8abd1c8c19693(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:40,845 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:40,845 INFO [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b., storeName=3d3705bd1d338a8a463a7f16f68aba1b/cf1, priority=9, startTime=1690185220773; duration=0sec 2023-07-24 07:53:40,846 INFO [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 3d3705bd1d338a8a463a7f16f68aba1b/cf2 of 3d3705bd1d338a8a463a7f16f68aba1b into 088f999024ec49dd851587c82e8d49d1(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:40,846 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:40,846 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:40,846 INFO [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b., storeName=3d3705bd1d338a8a463a7f16f68aba1b/cf2, priority=9, startTime=1690185220773; duration=0sec 2023-07-24 07:53:40,846 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:40,864 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:40,866 INFO [Listener at localhost/42537] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-07-24 07:53:40,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-07-24 07:53:40,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:40,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-24 07:53:40,870 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185220870"}]},"ts":"1690185220870"} 2023-07-24 07:53:40,871 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-07-24 07:53:40,873 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-07-24 07:53:40,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN}] 2023-07-24 07:53:40,874 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN 2023-07-24 07:53:40,875 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:40,875 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185220875"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185220875"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185220875"}]},"ts":"1690185220875"} 2023-07-24 07:53:40,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:53:40,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-24 07:53:41,028 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:41,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3d3705bd1d338a8a463a7f16f68aba1b, disabling compactions & flushes 2023-07-24 07:53:41,028 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:41,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:41,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. after waiting 0 ms 2023-07-24 07:53:41,028 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:41,029 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/2374a8a7f6e04140a3e5bb86774e3b34, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8e4a072eb3524db3bdcbb63b049ef808, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/db0043a832c240969ee9338f6467d4e8, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/f64401129d5c4e55b0b9a2cc1810da3f, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8689839085834b3786192c7de266a47c, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/da60249f6ea042ec9696767cfe028258] to archive 2023-07-24 07:53:41,030 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:41,032 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/2374a8a7f6e04140a3e5bb86774e3b34 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/2374a8a7f6e04140a3e5bb86774e3b34 2023-07-24 07:53:41,033 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8e4a072eb3524db3bdcbb63b049ef808 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8e4a072eb3524db3bdcbb63b049ef808 2023-07-24 07:53:41,034 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/1f2bfd8d5a3f425481a64f68c4a5aedd 2023-07-24 07:53:41,035 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/db0043a832c240969ee9338f6467d4e8 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/db0043a832c240969ee9338f6467d4e8 2023-07-24 07:53:41,037 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/f64401129d5c4e55b0b9a2cc1810da3f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/f64401129d5c4e55b0b9a2cc1810da3f 2023-07-24 07:53:41,038 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8689839085834b3786192c7de266a47c to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8689839085834b3786192c7de266a47c 2023-07-24 07:53:41,039 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/da60249f6ea042ec9696767cfe028258 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/da60249f6ea042ec9696767cfe028258 2023-07-24 07:53:41,044 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/e94ae2acbfc74575955d2f49aeb021fe, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/076e78f8f9a749e5ab1e1aa364f5df5d, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/45953d4fca6844a383912ca2af35d812, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8c1f3e287b2f49e2bb864bffb376cb77, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/48a5f3335cb94368bd6f70431a5d7905, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8b846816768149f79c18764b02685d99, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/69f7c7b9072b41f8a76def5515e9b81c] to archive 2023-07-24 07:53:41,045 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:41,047 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/e94ae2acbfc74575955d2f49aeb021fe to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/e94ae2acbfc74575955d2f49aeb021fe 2023-07-24 07:53:41,048 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/076e78f8f9a749e5ab1e1aa364f5df5d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/076e78f8f9a749e5ab1e1aa364f5df5d 2023-07-24 07:53:41,049 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/45953d4fca6844a383912ca2af35d812 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/45953d4fca6844a383912ca2af35d812 2023-07-24 07:53:41,050 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8c1f3e287b2f49e2bb864bffb376cb77 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8c1f3e287b2f49e2bb864bffb376cb77 2023-07-24 07:53:41,052 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/48a5f3335cb94368bd6f70431a5d7905 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/48a5f3335cb94368bd6f70431a5d7905 2023-07-24 07:53:41,053 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8b846816768149f79c18764b02685d99 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/8b846816768149f79c18764b02685d99 2023-07-24 07:53:41,054 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/69f7c7b9072b41f8a76def5515e9b81c to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/69f7c7b9072b41f8a76def5515e9b81c 2023-07-24 07:53:41,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 07:53:41,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:41,059 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b. 2023-07-24 07:53:41,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3d3705bd1d338a8a463a7f16f68aba1b: 2023-07-24 07:53:41,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:41,061 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=3d3705bd1d338a8a463a7f16f68aba1b, regionState=CLOSED 2023-07-24 07:53:41,061 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185221061"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185221061"}]},"ts":"1690185221061"} 2023-07-24 07:53:41,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-07-24 07:53:41,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 3d3705bd1d338a8a463a7f16f68aba1b, server=jenkins-hbase4.apache.org,46877,1690185199045 in 187 msec 2023-07-24 07:53:41,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-07-24 07:53:41,067 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, UNASSIGN in 192 msec 2023-07-24 07:53:41,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185221068"}]},"ts":"1690185221068"} 2023-07-24 07:53:41,069 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-07-24 07:53:41,070 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-07-24 07:53:41,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 205 msec 2023-07-24 07:53:41,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-24 07:53:41,172 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-07-24 07:53:41,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-07-24 07:53:41,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:41,176 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:41,177 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:41,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-07-24 07:53:41,180 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:41,182 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/recovered.edits] 2023-07-24 07:53:41,185 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8df97f45d2af45c39cd8abd1c8c19693 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf1/8df97f45d2af45c39cd8abd1c8c19693 2023-07-24 07:53:41,188 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/088f999024ec49dd851587c82e8d49d1 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/cf2/088f999024ec49dd851587c82e8d49d1 2023-07-24 07:53:41,190 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/recovered.edits/90.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b/recovered.edits/90.seqid 2023-07-24 07:53:41,191 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackAndDoubleExecution/3d3705bd1d338a8a463a7f16f68aba1b 2023-07-24 07:53:41,191 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-07-24 07:53:41,194 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:41,196 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-07-24 07:53:41,198 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-07-24 07:53:41,199 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:41,199 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-07-24 07:53:41,199 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185221199"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:41,200 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-24 07:53:41,200 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 3d3705bd1d338a8a463a7f16f68aba1b, NAME => 'testRollbackAndDoubleExecution,,1690185213311.3d3705bd1d338a8a463a7f16f68aba1b.', STARTKEY => '', ENDKEY => ''}] 2023-07-24 07:53:41,200 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-07-24 07:53:41,201 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690185221200"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:41,202 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-07-24 07:53:41,204 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:53:41,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 31 msec 2023-07-24 07:53:41,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-07-24 07:53:41,279 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-07-24 07:53:41,295 INFO [Listener at localhost/42537] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=405 (was 402) Potentially hanging thread: RS:0;jenkins-hbase4:46877-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:51896 [Waiting for operation #48] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:49106 [Waiting for operation #37] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:37298 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:37246 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:51824 [Waiting for operation #47] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:37226 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:49306 [Waiting for operation #34] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:59280 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:51926 [Waiting for operation #42] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:38958 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:49150 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:49108 [Waiting for operation #30] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:37258 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:59266 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:59282 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2056090878_17 at /127.0.0.1:36978 [Waiting for operation #39] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=862 (was 852) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=318 (was 330), ProcessCount=174 (was 174), AvailableMemoryMB=8919 (was 9033) 2023-07-24 07:53:41,309 INFO [Listener at localhost/42537] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=405, OpenFileDescriptor=862, MaxFileDescriptor=60000, SystemLoadAverage=318, ProcessCount=174, AvailableMemoryMB=8919 2023-07-24 07:53:41,310 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:41,313 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 07:53:41,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 07:53:41,314 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:53:41,316 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 07:53:41,317 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 07:53:41,319 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,319 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3 empty. 2023-07-24 07:53:41,320 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,320 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-07-24 07:53:41,330 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-07-24 07:53:41,331 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => df88344c1a5d2a50942a8f97222251c3, NAME => 'testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp 2023-07-24 07:53:41,346 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:41,347 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing df88344c1a5d2a50942a8f97222251c3, disabling compactions & flushes 2023-07-24 07:53:41,347 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. 2023-07-24 07:53:41,347 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. 2023-07-24 07:53:41,347 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. after waiting 0 ms 2023-07-24 07:53:41,347 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. 2023-07-24 07:53:41,347 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. 2023-07-24 07:53:41,347 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for df88344c1a5d2a50942a8f97222251c3: 2023-07-24 07:53:41,352 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 07:53:41,353 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690185221353"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185221353"}]},"ts":"1690185221353"} 2023-07-24 07:53:41,355 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 07:53:41,356 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 07:53:41,356 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185221356"}]},"ts":"1690185221356"} 2023-07-24 07:53:41,357 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-07-24 07:53:41,360 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:53:41,362 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:53:41,362 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:53:41,362 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:53:41,362 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:53:41,362 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=df88344c1a5d2a50942a8f97222251c3, ASSIGN}] 2023-07-24 07:53:41,364 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=df88344c1a5d2a50942a8f97222251c3, ASSIGN 2023-07-24 07:53:41,364 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=df88344c1a5d2a50942a8f97222251c3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:53:41,514 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:53:41,516 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=df88344c1a5d2a50942a8f97222251c3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:41,516 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690185221515"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185221515"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185221515"}]},"ts":"1690185221515"} 2023-07-24 07:53:41,518 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure df88344c1a5d2a50942a8f97222251c3, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:41,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. 2023-07-24 07:53:41,672 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => df88344c1a5d2a50942a8f97222251c3, NAME => 'testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:53:41,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:41,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:41,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,674 INFO [StoreOpener-df88344c1a5d2a50942a8f97222251c3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,676 DEBUG [StoreOpener-df88344c1a5d2a50942a8f97222251c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3/cf1 2023-07-24 07:53:41,676 DEBUG [StoreOpener-df88344c1a5d2a50942a8f97222251c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3/cf1 2023-07-24 07:53:41,676 INFO [StoreOpener-df88344c1a5d2a50942a8f97222251c3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region df88344c1a5d2a50942a8f97222251c3 columnFamilyName cf1 2023-07-24 07:53:41,677 INFO [StoreOpener-df88344c1a5d2a50942a8f97222251c3-1] regionserver.HStore(310): Store=df88344c1a5d2a50942a8f97222251c3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:41,677 INFO [StoreOpener-df88344c1a5d2a50942a8f97222251c3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,679 DEBUG [StoreOpener-df88344c1a5d2a50942a8f97222251c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3/cf2 2023-07-24 07:53:41,679 DEBUG [StoreOpener-df88344c1a5d2a50942a8f97222251c3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3/cf2 2023-07-24 07:53:41,679 INFO [StoreOpener-df88344c1a5d2a50942a8f97222251c3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region df88344c1a5d2a50942a8f97222251c3 columnFamilyName cf2 2023-07-24 07:53:41,680 INFO [StoreOpener-df88344c1a5d2a50942a8f97222251c3-1] regionserver.HStore(310): Store=df88344c1a5d2a50942a8f97222251c3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:41,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:41,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:53:41,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened df88344c1a5d2a50942a8f97222251c3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66678147, jitterRate=-0.006418183445930481}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:41,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for df88344c1a5d2a50942a8f97222251c3: 2023-07-24 07:53:41,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3., pid=55, masterSystemTime=1690185221669 2023-07-24 07:53:41,691 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. 2023-07-24 07:53:41,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. 2023-07-24 07:53:41,691 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=df88344c1a5d2a50942a8f97222251c3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:41,691 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690185221691"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185221691"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185221691"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185221691"}]},"ts":"1690185221691"} 2023-07-24 07:53:41,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-07-24 07:53:41,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure df88344c1a5d2a50942a8f97222251c3, server=jenkins-hbase4.apache.org,46543,1690185199091 in 175 msec 2023-07-24 07:53:41,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-07-24 07:53:41,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=df88344c1a5d2a50942a8f97222251c3, ASSIGN in 333 msec 2023-07-24 07:53:41,698 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 07:53:41,699 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185221698"}]},"ts":"1690185221698"} 2023-07-24 07:53:41,700 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-07-24 07:53:41,702 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 07:53:41,704 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 388 msec 2023-07-24 07:53:41,816 DEBUG [Listener at localhost/42537] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:41,817 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:41,818 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:41,818 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:41,818 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:41,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=df88344c1a5d2a50942a8f97222251c3, UNASSIGN}] 2023-07-24 07:53:41,823 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=df88344c1a5d2a50942a8f97222251c3, UNASSIGN 2023-07-24 07:53:41,824 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=df88344c1a5d2a50942a8f97222251c3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:41,824 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690185221824"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185221824"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185221824"}]},"ts":"1690185221824"} 2023-07-24 07:53:41,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure df88344c1a5d2a50942a8f97222251c3, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:41,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,979 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing df88344c1a5d2a50942a8f97222251c3, disabling compactions & flushes 2023-07-24 07:53:41,979 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. 2023-07-24 07:53:41,979 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. 2023-07-24 07:53:41,979 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. after waiting 0 ms 2023-07-24 07:53:41,979 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. 2023-07-24 07:53:41,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 07:53:41,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:41,984 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3. 2023-07-24 07:53:41,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for df88344c1a5d2a50942a8f97222251c3: 2023-07-24 07:53:41,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:41,986 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=df88344c1a5d2a50942a8f97222251c3, regionState=CLOSED 2023-07-24 07:53:41,986 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690185221986"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185221986"}]},"ts":"1690185221986"} 2023-07-24 07:53:41,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-07-24 07:53:41,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure df88344c1a5d2a50942a8f97222251c3, server=jenkins-hbase4.apache.org,46543,1690185199091 in 162 msec 2023-07-24 07:53:41,990 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-07-24 07:53:41,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=df88344c1a5d2a50942a8f97222251c3, UNASSIGN in 167 msec 2023-07-24 07:53:42,016 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-07-24 07:53:42,018 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-07-24 07:53:42,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690185222019"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1690185222019"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1690185222019"}]},"ts":"1690185222019"} 2023-07-24 07:53:42,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185222019"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185222019"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185222019"}]},"ts":"1690185222019"} 2023-07-24 07:53:42,020 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185222019"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185222019"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185222019"}]},"ts":"1690185222019"} 2023-07-24 07:53:42,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=950a2f2cad20e0dd1edc0ff5cb3fe097, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47fb575219491f8a6dfebbe5390ced75, ASSIGN}] 2023-07-24 07:53:42,026 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47fb575219491f8a6dfebbe5390ced75, ASSIGN 2023-07-24 07:53:42,027 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47fb575219491f8a6dfebbe5390ced75, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:53:42,027 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=950a2f2cad20e0dd1edc0ff5cb3fe097, ASSIGN 2023-07-24 07:53:42,029 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=950a2f2cad20e0dd1edc0ff5cb3fe097, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:53:42,177 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 07:53:42,179 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=950a2f2cad20e0dd1edc0ff5cb3fe097, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:42,179 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185222179"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185222179"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185222179"}]},"ts":"1690185222179"} 2023-07-24 07:53:42,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 950a2f2cad20e0dd1edc0ff5cb3fe097, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:42,184 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=47fb575219491f8a6dfebbe5390ced75, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:42,184 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185222184"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185222184"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185222184"}]},"ts":"1690185222184"} 2023-07-24 07:53:42,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 47fb575219491f8a6dfebbe5390ced75, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:42,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75. 2023-07-24 07:53:42,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 47fb575219491f8a6dfebbe5390ced75, NAME => 'testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75.', STARTKEY => '41', ENDKEY => ''} 2023-07-24 07:53:42,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:42,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:42,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,338 INFO [StoreOpener-47fb575219491f8a6dfebbe5390ced75-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,339 DEBUG [StoreOpener-47fb575219491f8a6dfebbe5390ced75-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75/cf1 2023-07-24 07:53:42,339 DEBUG [StoreOpener-47fb575219491f8a6dfebbe5390ced75-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75/cf1 2023-07-24 07:53:42,340 INFO [StoreOpener-47fb575219491f8a6dfebbe5390ced75-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47fb575219491f8a6dfebbe5390ced75 columnFamilyName cf1 2023-07-24 07:53:42,340 INFO [StoreOpener-47fb575219491f8a6dfebbe5390ced75-1] regionserver.HStore(310): Store=47fb575219491f8a6dfebbe5390ced75/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:42,340 INFO [StoreOpener-47fb575219491f8a6dfebbe5390ced75-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,342 DEBUG [StoreOpener-47fb575219491f8a6dfebbe5390ced75-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75/cf2 2023-07-24 07:53:42,342 DEBUG [StoreOpener-47fb575219491f8a6dfebbe5390ced75-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75/cf2 2023-07-24 07:53:42,342 INFO [StoreOpener-47fb575219491f8a6dfebbe5390ced75-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 47fb575219491f8a6dfebbe5390ced75 columnFamilyName cf2 2023-07-24 07:53:42,343 INFO [StoreOpener-47fb575219491f8a6dfebbe5390ced75-1] regionserver.HStore(310): Store=47fb575219491f8a6dfebbe5390ced75/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:42,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,345 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,346 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:42,348 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 47fb575219491f8a6dfebbe5390ced75; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61070237, jitterRate=-0.08998255431652069}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:42,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 47fb575219491f8a6dfebbe5390ced75: 2023-07-24 07:53:42,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75., pid=62, masterSystemTime=1690185222333 2023-07-24 07:53:42,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75. 2023-07-24 07:53:42,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75. 2023-07-24 07:53:42,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097. 2023-07-24 07:53:42,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 950a2f2cad20e0dd1edc0ff5cb3fe097, NAME => 'testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097.', STARTKEY => '', ENDKEY => '41'} 2023-07-24 07:53:42,352 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=47fb575219491f8a6dfebbe5390ced75, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:42,352 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185222352"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185222352"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185222352"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185222352"}]},"ts":"1690185222352"} 2023-07-24 07:53:42,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:42,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:42,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,354 INFO [StoreOpener-950a2f2cad20e0dd1edc0ff5cb3fe097-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-07-24 07:53:42,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 47fb575219491f8a6dfebbe5390ced75, server=jenkins-hbase4.apache.org,46543,1690185199091 in 168 msec 2023-07-24 07:53:42,356 DEBUG [StoreOpener-950a2f2cad20e0dd1edc0ff5cb3fe097-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097/cf1 2023-07-24 07:53:42,356 DEBUG [StoreOpener-950a2f2cad20e0dd1edc0ff5cb3fe097-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097/cf1 2023-07-24 07:53:42,356 INFO [StoreOpener-950a2f2cad20e0dd1edc0ff5cb3fe097-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 950a2f2cad20e0dd1edc0ff5cb3fe097 columnFamilyName cf1 2023-07-24 07:53:42,357 INFO [StoreOpener-950a2f2cad20e0dd1edc0ff5cb3fe097-1] regionserver.HStore(310): Store=950a2f2cad20e0dd1edc0ff5cb3fe097/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:42,357 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47fb575219491f8a6dfebbe5390ced75, ASSIGN in 331 msec 2023-07-24 07:53:42,357 INFO [StoreOpener-950a2f2cad20e0dd1edc0ff5cb3fe097-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,359 DEBUG [StoreOpener-950a2f2cad20e0dd1edc0ff5cb3fe097-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097/cf2 2023-07-24 07:53:42,359 DEBUG [StoreOpener-950a2f2cad20e0dd1edc0ff5cb3fe097-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097/cf2 2023-07-24 07:53:42,359 INFO [StoreOpener-950a2f2cad20e0dd1edc0ff5cb3fe097-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 950a2f2cad20e0dd1edc0ff5cb3fe097 columnFamilyName cf2 2023-07-24 07:53:42,359 INFO [StoreOpener-950a2f2cad20e0dd1edc0ff5cb3fe097-1] regionserver.HStore(310): Store=950a2f2cad20e0dd1edc0ff5cb3fe097/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:42,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:42,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 950a2f2cad20e0dd1edc0ff5cb3fe097; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65098491, jitterRate=-0.029956892132759094}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:42,366 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 950a2f2cad20e0dd1edc0ff5cb3fe097: 2023-07-24 07:53:42,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097., pid=61, masterSystemTime=1690185222333 2023-07-24 07:53:42,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097. 2023-07-24 07:53:42,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097. 2023-07-24 07:53:42,369 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=950a2f2cad20e0dd1edc0ff5cb3fe097, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:42,369 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185222369"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185222369"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185222369"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185222369"}]},"ts":"1690185222369"} 2023-07-24 07:53:42,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-07-24 07:53:42,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 950a2f2cad20e0dd1edc0ff5cb3fe097, server=jenkins-hbase4.apache.org,46543,1690185199091 in 189 msec 2023-07-24 07:53:42,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-07-24 07:53:42,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=950a2f2cad20e0dd1edc0ff5cb3fe097, ASSIGN in 347 msec 2023-07-24 07:53:42,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 in 558 msec 2023-07-24 07:53:42,572 DEBUG [Listener at localhost/42537] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 07:53:42,575 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:42,577 INFO [Listener at localhost/42537] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-07-24 07:53:42,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-07-24 07:53:42,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:53:42,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-24 07:53:42,581 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185222581"}]},"ts":"1690185222581"} 2023-07-24 07:53:42,583 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-07-24 07:53:42,585 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-07-24 07:53:42,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=950a2f2cad20e0dd1edc0ff5cb3fe097, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47fb575219491f8a6dfebbe5390ced75, UNASSIGN}] 2023-07-24 07:53:42,586 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47fb575219491f8a6dfebbe5390ced75, UNASSIGN 2023-07-24 07:53:42,587 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=47fb575219491f8a6dfebbe5390ced75, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:42,587 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185222587"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185222587"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185222587"}]},"ts":"1690185222587"} 2023-07-24 07:53:42,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 47fb575219491f8a6dfebbe5390ced75, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:42,591 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=950a2f2cad20e0dd1edc0ff5cb3fe097, UNASSIGN 2023-07-24 07:53:42,592 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=950a2f2cad20e0dd1edc0ff5cb3fe097, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:42,592 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185222592"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185222592"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185222592"}]},"ts":"1690185222592"} 2023-07-24 07:53:42,593 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 950a2f2cad20e0dd1edc0ff5cb3fe097, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:42,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-24 07:53:42,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 950a2f2cad20e0dd1edc0ff5cb3fe097, disabling compactions & flushes 2023-07-24 07:53:42,742 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097. 2023-07-24 07:53:42,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097. 2023-07-24 07:53:42,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097. after waiting 0 ms 2023-07-24 07:53:42,742 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097. 2023-07-24 07:53:42,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-24 07:53:42,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:42,747 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097. 2023-07-24 07:53:42,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 950a2f2cad20e0dd1edc0ff5cb3fe097: 2023-07-24 07:53:42,749 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,749 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,750 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 47fb575219491f8a6dfebbe5390ced75, disabling compactions & flushes 2023-07-24 07:53:42,750 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75. 2023-07-24 07:53:42,750 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75. 2023-07-24 07:53:42,750 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75. after waiting 0 ms 2023-07-24 07:53:42,750 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75. 2023-07-24 07:53:42,750 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=950a2f2cad20e0dd1edc0ff5cb3fe097, regionState=CLOSED 2023-07-24 07:53:42,750 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185222750"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185222750"}]},"ts":"1690185222750"} 2023-07-24 07:53:42,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-24 07:53:42,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-07-24 07:53:42,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 950a2f2cad20e0dd1edc0ff5cb3fe097, server=jenkins-hbase4.apache.org,46543,1690185199091 in 159 msec 2023-07-24 07:53:42,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:42,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75. 2023-07-24 07:53:42,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 47fb575219491f8a6dfebbe5390ced75: 2023-07-24 07:53:42,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=950a2f2cad20e0dd1edc0ff5cb3fe097, UNASSIGN in 169 msec 2023-07-24 07:53:42,756 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,756 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=47fb575219491f8a6dfebbe5390ced75, regionState=CLOSED 2023-07-24 07:53:42,756 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185222756"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185222756"}]},"ts":"1690185222756"} 2023-07-24 07:53:42,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-07-24 07:53:42,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 47fb575219491f8a6dfebbe5390ced75, server=jenkins-hbase4.apache.org,46543,1690185199091 in 169 msec 2023-07-24 07:53:42,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-07-24 07:53:42,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=47fb575219491f8a6dfebbe5390ced75, UNASSIGN in 174 msec 2023-07-24 07:53:42,761 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185222761"}]},"ts":"1690185222761"} 2023-07-24 07:53:42,762 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-07-24 07:53:42,764 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-07-24 07:53:42,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 186 msec 2023-07-24 07:53:42,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-24 07:53:42,884 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-07-24 07:53:42,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-07-24 07:53:42,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:53:42,888 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:53:42,888 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:53:42,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-24 07:53:42,894 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:42,894 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,894 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,896 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75/recovered.edits] 2023-07-24 07:53:42,896 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3/recovered.edits] 2023-07-24 07:53:42,896 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097/recovered.edits] 2023-07-24 07:53:42,903 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75/recovered.edits/7.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75/recovered.edits/7.seqid 2023-07-24 07:53:42,903 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3/recovered.edits/4.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3/recovered.edits/4.seqid 2023-07-24 07:53:42,903 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097/recovered.edits/7.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097/recovered.edits/7.seqid 2023-07-24 07:53:42,904 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:53:42,904 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/df88344c1a5d2a50942a8f97222251c3 2023-07-24 07:53:42,904 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionNoStoreFile/950a2f2cad20e0dd1edc0ff5cb3fe097 2023-07-24 07:53:42,904 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-07-24 07:53:42,907 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:53:42,908 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-07-24 07:53:42,910 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-07-24 07:53:42,911 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:53:42,911 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-07-24 07:53:42,911 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185222911"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:42,912 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185222911"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:42,912 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185222911"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:42,913 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 07:53:42,913 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => df88344c1a5d2a50942a8f97222251c3, NAME => 'testSplitTableRegionNoStoreFile,,1690185221313.df88344c1a5d2a50942a8f97222251c3.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 950a2f2cad20e0dd1edc0ff5cb3fe097, NAME => 'testSplitTableRegionNoStoreFile,,1690185221816.950a2f2cad20e0dd1edc0ff5cb3fe097.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 47fb575219491f8a6dfebbe5390ced75, NAME => 'testSplitTableRegionNoStoreFile,41,1690185221816.47fb575219491f8a6dfebbe5390ced75.', STARTKEY => '41', ENDKEY => ''}] 2023-07-24 07:53:42,913 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-07-24 07:53:42,914 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690185222913"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:42,915 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-07-24 07:53:42,917 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:53:42,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 32 msec 2023-07-24 07:53:42,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-24 07:53:42,990 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-07-24 07:53:43,005 INFO [Listener at localhost/42537] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=409 (was 405) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-236827379_17 at /127.0.0.1:49306 [Waiting for operation #35] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-236827379_17 at /127.0.0.1:38958 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=862 (was 862), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=318 (was 318), ProcessCount=174 (was 174), AvailableMemoryMB=8918 (was 8919) 2023-07-24 07:53:43,019 INFO [Listener at localhost/42537] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=409, OpenFileDescriptor=862, MaxFileDescriptor=60000, SystemLoadAverage=318, ProcessCount=174, AvailableMemoryMB=8917 2023-07-24 07:53:43,019 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:43,023 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 07:53:43,023 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 07:53:43,024 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,026 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 07:53:43,027 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 07:53:43,028 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:43,029 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356 empty. 2023-07-24 07:53:43,029 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:43,029 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-07-24 07:53:43,044 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-07-24 07:53:43,045 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => d0b3800d667beee4d07b06d26530b356, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp 2023-07-24 07:53:43,058 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:43,058 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing d0b3800d667beee4d07b06d26530b356, disabling compactions & flushes 2023-07-24 07:53:43,058 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,058 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,058 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. after waiting 0 ms 2023-07-24 07:53:43,058 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,058 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,058 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:43,060 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 07:53:43,061 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690185223061"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185223061"}]},"ts":"1690185223061"} 2023-07-24 07:53:43,062 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 07:53:43,063 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 07:53:43,063 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185223063"}]},"ts":"1690185223063"} 2023-07-24 07:53:43,064 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-07-24 07:53:43,067 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:53:43,067 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:53:43,067 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:53:43,067 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:53:43,067 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:53:43,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d0b3800d667beee4d07b06d26530b356, ASSIGN}] 2023-07-24 07:53:43,069 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d0b3800d667beee4d07b06d26530b356, ASSIGN 2023-07-24 07:53:43,070 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d0b3800d667beee4d07b06d26530b356, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:53:43,220 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:53:43,221 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=d0b3800d667beee4d07b06d26530b356, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,221 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690185223221"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185223221"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185223221"}]},"ts":"1690185223221"} 2023-07-24 07:53:43,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure d0b3800d667beee4d07b06d26530b356, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:43,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d0b3800d667beee4d07b06d26530b356, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:53:43,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:43,378 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:43,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:43,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:43,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:43,380 INFO [StoreOpener-d0b3800d667beee4d07b06d26530b356-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:43,381 DEBUG [StoreOpener-d0b3800d667beee4d07b06d26530b356-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1 2023-07-24 07:53:43,381 DEBUG [StoreOpener-d0b3800d667beee4d07b06d26530b356-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1 2023-07-24 07:53:43,382 INFO [StoreOpener-d0b3800d667beee4d07b06d26530b356-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0b3800d667beee4d07b06d26530b356 columnFamilyName cf1 2023-07-24 07:53:43,382 INFO [StoreOpener-d0b3800d667beee4d07b06d26530b356-1] regionserver.HStore(310): Store=d0b3800d667beee4d07b06d26530b356/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:43,382 INFO [StoreOpener-d0b3800d667beee4d07b06d26530b356-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:43,384 DEBUG [StoreOpener-d0b3800d667beee4d07b06d26530b356-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2 2023-07-24 07:53:43,384 DEBUG [StoreOpener-d0b3800d667beee4d07b06d26530b356-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2 2023-07-24 07:53:43,384 INFO [StoreOpener-d0b3800d667beee4d07b06d26530b356-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0b3800d667beee4d07b06d26530b356 columnFamilyName cf2 2023-07-24 07:53:43,385 INFO [StoreOpener-d0b3800d667beee4d07b06d26530b356-1] regionserver.HStore(310): Store=d0b3800d667beee4d07b06d26530b356/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:43,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:43,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:43,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:43,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:43,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:53:43,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d0b3800d667beee4d07b06d26530b356; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60528787, jitterRate=-0.0980507880449295}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:43,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:43,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356., pid=71, masterSystemTime=1690185223375 2023-07-24 07:53:43,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,396 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=d0b3800d667beee4d07b06d26530b356, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,396 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690185223396"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185223396"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185223396"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185223396"}]},"ts":"1690185223396"} 2023-07-24 07:53:43,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-07-24 07:53:43,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure d0b3800d667beee4d07b06d26530b356, server=jenkins-hbase4.apache.org,46543,1690185199091 in 174 msec 2023-07-24 07:53:43,400 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-07-24 07:53:43,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d0b3800d667beee4d07b06d26530b356, ASSIGN in 331 msec 2023-07-24 07:53:43,401 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 07:53:43,401 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185223401"}]},"ts":"1690185223401"} 2023-07-24 07:53:43,402 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-07-24 07:53:43,405 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 07:53:43,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 382 msec 2023-07-24 07:53:43,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:43,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,534 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:43,535 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,535 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 07:53:43,535 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 07:53:43,535 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,535 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,537 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,537 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,537 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,537 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,538 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,538 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:43,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,539 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:43,539 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:43,539 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,539 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,539 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:43,539 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:43,540 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,540 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. started... 2023-07-24 07:53:43,541 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0b3800d667beee4d07b06d26530b356 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 07:53:43,542 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,543 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,543 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,543 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:43,543 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,543 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:43,551 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/2858f8ef6d884085989a3a44e5c3bd31 2023-07-24 07:53:43,564 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/10c4507131c5422e8dfbfd2ba643d5fc 2023-07-24 07:53:43,569 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/2858f8ef6d884085989a3a44e5c3bd31 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31 2023-07-24 07:53:43,573 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:53:43,574 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/10c4507131c5422e8dfbfd2ba643d5fc as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc 2023-07-24 07:53:43,578 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:53:43,579 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d0b3800d667beee4d07b06d26530b356 in 38ms, sequenceid=6, compaction requested=false 2023-07-24 07:53:43,579 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:43,580 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:43,580 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,580 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:43,580 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:43,580 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,580 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:43,580 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:43,581 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,581 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:43,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:43,582 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,582 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:43,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:43,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:43,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:43,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 07:53:43,584 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@509ad831[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:43,584 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:43,584 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,586 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,586 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,586 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,586 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:43,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,586 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:43,586 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:43,586 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:43,586 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:43,586 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,587 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:43,586 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:43,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,587 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:43,587 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:43,587 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:43,588 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:43,588 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,588 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:43,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:43,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:43,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:43,589 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:43,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:43,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:43,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:43,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:43,592 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 07:53:43,592 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:43,592 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:43,592 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,592 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:43,594 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,594 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,594 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,594 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:43,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,594 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,594 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,594 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,595 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:43,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,595 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:43,596 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,597 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,598 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,598 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:43,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,610 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,610 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,610 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,610 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,610 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,610 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,610 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,610 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,610 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,610 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:43,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:43,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,610 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,611 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,610 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,611 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,611 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,611 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,611 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,611 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,613 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:43,613 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:43,713 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:43,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:43,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:43,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:43,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,730 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:43,730 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,730 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 07:53:43,730 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 07:53:43,731 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,731 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,732 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,732 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,732 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,732 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,733 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,733 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:43,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,734 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,734 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:43,734 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,734 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:43,734 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:43,734 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,734 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,734 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:43,734 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:43,734 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:43,735 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:43,734 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,735 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:43,735 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:43,735 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:43,735 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,735 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,735 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. started... 2023-07-24 07:53:43,735 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:43,735 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:43,735 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:43,735 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0b3800d667beee4d07b06d26530b356 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:43,736 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,737 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:43,744 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/b33768ad46944051b4c5e938ec335c49 2023-07-24 07:53:43,756 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/3c41ef91fb684a13a915be1ef4405836 2023-07-24 07:53:43,761 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/b33768ad46944051b4c5e938ec335c49 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49 2023-07-24 07:53:43,764 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:53:43,765 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/3c41ef91fb684a13a915be1ef4405836 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836 2023-07-24 07:53:43,769 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:53:43,770 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d0b3800d667beee4d07b06d26530b356 in 35ms, sequenceid=19, compaction requested=false 2023-07-24 07:53:43,770 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:43,770 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,770 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:43,770 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:43,770 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,770 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:43,770 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:43,776 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,777 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:43,777 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,777 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:43,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:43,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 07:53:43,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52ebcb69[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:43,779 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:43,779 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,780 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,780 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,780 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:43,780 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,781 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:43,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:43,780 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:43,781 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:43,782 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,782 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:43,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:43,782 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,782 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:43,783 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:43,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:43,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:43,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:43,785 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 07:53:43,785 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:43,785 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:43,785 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,785 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:43,788 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,788 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,788 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,788 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:43,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,788 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,788 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,788 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,789 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:43,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,789 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,790 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:43,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,791 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:43,792 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,792 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,795 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,795 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,795 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,795 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,795 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,795 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,795 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:43,795 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:43,795 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,795 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,795 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,795 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,796 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,795 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,796 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:43,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,796 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,796 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,797 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,797 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,797 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,797 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,797 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,896 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:43,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:43,915 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:43,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:43,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,917 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:43,917 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,917 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 07:53:43,917 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 07:53:43,918 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,918 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,919 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,919 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,919 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,919 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,920 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,920 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:43,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:43,921 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,921 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,921 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:43,921 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,921 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:43,922 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:43,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,922 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:43,922 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:43,922 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. started... 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:43,922 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0b3800d667beee4d07b06d26530b356 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:43,922 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:43,924 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,925 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,925 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:43,927 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,927 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,927 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:43,937 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/f501e4b5972443a9a6d6b6c94d95004f 2023-07-24 07:53:43,949 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/d863ceea01a347ff80130486a58ce7ab 2023-07-24 07:53:43,954 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/f501e4b5972443a9a6d6b6c94d95004f as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f 2023-07-24 07:53:43,957 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:53:43,958 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/d863ceea01a347ff80130486a58ce7ab as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab 2023-07-24 07:53:43,962 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:53:43,963 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d0b3800d667beee4d07b06d26530b356 in 41ms, sequenceid=32, compaction requested=true 2023-07-24 07:53:43,963 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:43,963 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:43,963 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:43,963 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:43,963 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,963 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:43,963 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:43,969 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,969 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:43,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:43,970 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,970 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:43,972 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 07:53:43,972 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30c1de58[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:43,972 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:43,972 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,973 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,973 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,973 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,973 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:43,973 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:43,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,973 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,973 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:43,973 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:43,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,973 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:43,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,973 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:43,974 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:43,974 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:43,973 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:43,974 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:43,975 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:43,975 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,975 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:43,975 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:43,975 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,975 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:43,975 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:43,976 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:43,976 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,976 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:43,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:43,979 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 07:53:43,979 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:43,979 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:43,979 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,979 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:43,981 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,982 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,982 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,981 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:43,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:43,982 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,982 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,982 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:43,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,982 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,983 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:43,983 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:43,986 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,986 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,986 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,989 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,989 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,989 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,989 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,989 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,989 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:43,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,989 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,989 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:43,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,989 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:43,989 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:43,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:43,990 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:43,990 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,990 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:43,990 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,990 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:43,990 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:43,990 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:43,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:43,990 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:43,990 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:43,990 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,090 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:44,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:44,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:44,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:44,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,108 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:44,108 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,109 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 07:53:44,109 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 07:53:44,109 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,109 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,110 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,110 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,110 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,110 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,111 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,111 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,112 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,112 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,112 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,113 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,113 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,113 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,113 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,114 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,114 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,114 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,114 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,114 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,114 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,114 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,114 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,114 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,114 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,114 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,114 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:44,115 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:44,115 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. started... 2023-07-24 07:53:44,115 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0b3800d667beee4d07b06d26530b356 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,117 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,132 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/a753d07b8e934a76b252c6ba04351524 2023-07-24 07:53:44,148 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/d089ce7056b64a2eb5344e62ae81be65 2023-07-24 07:53:44,153 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/a753d07b8e934a76b252c6ba04351524 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/a753d07b8e934a76b252c6ba04351524 2023-07-24 07:53:44,158 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/a753d07b8e934a76b252c6ba04351524, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:53:44,159 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/d089ce7056b64a2eb5344e62ae81be65 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d089ce7056b64a2eb5344e62ae81be65 2023-07-24 07:53:44,164 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d089ce7056b64a2eb5344e62ae81be65, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:53:44,166 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d0b3800d667beee4d07b06d26530b356 in 51ms, sequenceid=45, compaction requested=true 2023-07-24 07:53:44,166 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:44,166 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:44,166 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:44,166 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:44,166 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,166 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,166 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,168 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,168 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,169 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,169 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,171 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 07:53:44,171 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2590cd4b[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:44,171 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:44,171 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,172 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,172 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,172 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,172 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,173 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,173 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,173 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,173 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,173 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,173 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,173 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,173 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,173 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:44,173 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:44,174 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,175 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,175 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,177 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,177 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,177 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,178 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,178 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,178 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,178 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:44,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:44,182 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 07:53:44,182 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:44,182 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:44,182 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,183 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:44,190 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,190 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,190 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,190 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,190 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,190 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,190 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,191 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,191 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,192 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,194 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,194 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,195 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,199 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,199 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,199 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,199 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,199 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,199 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,199 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:44,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:44,199 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,200 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:44,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,199 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,200 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,200 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,200 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,200 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,200 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,200 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,200 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,200 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,199 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,200 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:44,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,300 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:44,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:44,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:44,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:44,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,317 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:44,317 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,317 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 07:53:44,317 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 07:53:44,317 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,317 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,318 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,318 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,318 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,318 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,319 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,319 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:44,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,320 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,320 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,320 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,320 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,321 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,320 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,321 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,321 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,321 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,321 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. started... 2023-07-24 07:53:44,321 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,321 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0b3800d667beee4d07b06d26530b356 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:44,322 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,324 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,324 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,324 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,327 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,335 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/2d2a3c58f2fe460daca46281e9d9cb9c 2023-07-24 07:53:44,347 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/4d847fa15e084eda8112bde0ac92a620 2023-07-24 07:53:44,352 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/2d2a3c58f2fe460daca46281e9d9cb9c as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2d2a3c58f2fe460daca46281e9d9cb9c 2023-07-24 07:53:44,357 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2d2a3c58f2fe460daca46281e9d9cb9c, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:53:44,357 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/4d847fa15e084eda8112bde0ac92a620 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/4d847fa15e084eda8112bde0ac92a620 2023-07-24 07:53:44,362 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/4d847fa15e084eda8112bde0ac92a620, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:53:44,363 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d0b3800d667beee4d07b06d26530b356 in 42ms, sequenceid=58, compaction requested=true 2023-07-24 07:53:44,363 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:44,363 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:44,363 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:44,363 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:44,363 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,363 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,363 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,365 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,365 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,366 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,366 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,368 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 07:53:44,368 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23cdd298[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:44,368 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:44,368 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,369 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,369 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,369 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,369 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,369 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,369 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:44,369 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,370 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,370 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,370 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:44,370 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:44,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,370 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,370 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,370 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:44,371 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,371 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,371 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,372 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,372 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,372 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,372 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,372 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,372 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,372 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:44,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:44,377 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 07:53:44,377 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:44,377 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:44,377 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,377 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:44,380 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,380 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,380 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,380 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,380 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,380 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,380 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,381 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,382 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,382 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,384 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,384 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,385 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,387 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,387 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,387 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,387 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,387 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,387 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,387 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,388 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,388 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:44,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:44,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,387 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,388 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,388 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,388 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,388 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:44,388 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:44,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,388 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,389 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,389 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,389 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,389 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,488 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:44,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:44,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:44,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:44,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,504 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:44,504 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,504 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 07:53:44,504 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 07:53:44,504 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,504 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,508 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,508 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,508 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,508 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,509 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,509 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:44,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,511 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,511 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,510 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,511 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,511 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,511 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,511 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,511 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:44,511 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:44,511 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:44,511 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. started... 2023-07-24 07:53:44,511 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0b3800d667beee4d07b06d26530b356 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:53:44,513 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,514 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,514 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,514 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,514 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,514 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,520 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/55efa58a337544738a9b478b5e65ff05 2023-07-24 07:53:44,533 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/32de190c3385449084241048a0a316f3 2023-07-24 07:53:44,538 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/55efa58a337544738a9b478b5e65ff05 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/55efa58a337544738a9b478b5e65ff05 2023-07-24 07:53:44,542 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/55efa58a337544738a9b478b5e65ff05, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:53:44,542 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/32de190c3385449084241048a0a316f3 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/32de190c3385449084241048a0a316f3 2023-07-24 07:53:44,547 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/32de190c3385449084241048a0a316f3, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:53:44,548 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d0b3800d667beee4d07b06d26530b356 in 37ms, sequenceid=71, compaction requested=true 2023-07-24 07:53:44,548 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:44,548 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:44,548 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:44,548 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:44,548 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,548 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,548 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,549 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,550 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,550 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,550 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 07:53:44,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f71f50b[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:44,552 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:44,552 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,553 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,553 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,553 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,553 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,553 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,553 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,554 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:44,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:44,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:44,556 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,556 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,556 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,556 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:44,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:44,560 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 07:53:44,560 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:44,560 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:44,560 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,560 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:44,561 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,561 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,561 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,561 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,562 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,562 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,563 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,564 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,565 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,565 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:44,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:44,568 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:44,569 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:44,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,568 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,569 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,569 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,569 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,569 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,569 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,569 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,669 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:44,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:44,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:44,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:44,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,683 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:44,683 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,683 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 07:53:44,683 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 07:53:44,684 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,684 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,686 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,686 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,686 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,686 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,686 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,686 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:44,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,687 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,687 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,687 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,687 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,687 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,688 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,688 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,688 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:44,688 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:44,688 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:44,688 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. started... 2023-07-24 07:53:44,688 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0b3800d667beee4d07b06d26530b356 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-07-24 07:53:44,689 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,689 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,690 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,690 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,690 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,690 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,701 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/9283b60dca114095a8dace3728aa08ac 2023-07-24 07:53:44,705 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9283b60dca114095a8dace3728aa08ac 2023-07-24 07:53:44,714 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/075b9f5882fc41758f6c8ed1e424dedd 2023-07-24 07:53:44,717 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 075b9f5882fc41758f6c8ed1e424dedd 2023-07-24 07:53:44,718 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/9283b60dca114095a8dace3728aa08ac as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac 2023-07-24 07:53:44,722 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9283b60dca114095a8dace3728aa08ac 2023-07-24 07:53:44,722 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac, entries=9, sequenceid=83, filesize=5.1 K 2023-07-24 07:53:44,722 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/075b9f5882fc41758f6c8ed1e424dedd as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd 2023-07-24 07:53:44,726 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 075b9f5882fc41758f6c8ed1e424dedd 2023-07-24 07:53:44,726 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd, entries=9, sequenceid=83, filesize=5.1 K 2023-07-24 07:53:44,727 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for d0b3800d667beee4d07b06d26530b356 in 39ms, sequenceid=83, compaction requested=true 2023-07-24 07:53:44,727 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:44,727 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,729 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,729 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,729 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,729 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 07:53:44,731 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@117ccda3[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:44,731 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:44,731 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,732 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,732 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,732 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,733 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,733 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,733 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:44,733 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,733 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,733 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,733 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,733 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,733 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,733 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:44,733 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:44,735 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,736 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,736 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,739 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,739 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,739 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,739 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,739 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,739 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,739 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:44,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:44,742 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 07:53:44,742 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:44,743 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:44,743 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,743 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:44,744 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,744 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,744 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,744 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,744 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,744 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,745 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,745 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,745 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,745 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,747 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,747 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,748 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,751 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,751 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,751 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,751 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,751 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,751 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,751 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,751 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,751 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:44,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:44,751 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,751 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,752 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,752 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:44,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,752 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,752 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:44,752 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,752 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,752 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,752 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,753 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,852 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:44,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:44,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:44,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:44,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,863 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:44,864 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,864 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 07:53:44,864 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 07:53:44,864 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,864 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,865 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,865 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,865 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,865 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,866 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,866 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:44,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,866 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,867 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,867 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,867 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,867 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,867 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,867 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:44,867 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,867 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,867 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,868 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:44,868 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:44,868 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,868 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,868 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:44,868 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:44,868 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:44,868 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:44,868 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. started... 2023-07-24 07:53:44,868 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0b3800d667beee4d07b06d26530b356 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-07-24 07:53:44,869 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,869 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,869 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,869 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,870 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,870 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,881 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/c63cb36288504968821e1cf4b9da9e51 2023-07-24 07:53:44,885 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c63cb36288504968821e1cf4b9da9e51 2023-07-24 07:53:44,893 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/27d030625e774ccd81013befa296e9ea 2023-07-24 07:53:44,897 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27d030625e774ccd81013befa296e9ea 2023-07-24 07:53:44,897 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/c63cb36288504968821e1cf4b9da9e51 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/c63cb36288504968821e1cf4b9da9e51 2023-07-24 07:53:44,900 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c63cb36288504968821e1cf4b9da9e51 2023-07-24 07:53:44,901 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/c63cb36288504968821e1cf4b9da9e51, entries=5, sequenceid=91, filesize=5.0 K 2023-07-24 07:53:44,901 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/27d030625e774ccd81013befa296e9ea as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/27d030625e774ccd81013befa296e9ea 2023-07-24 07:53:44,905 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27d030625e774ccd81013befa296e9ea 2023-07-24 07:53:44,905 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/27d030625e774ccd81013befa296e9ea, entries=5, sequenceid=91, filesize=5.0 K 2023-07-24 07:53:44,905 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for d0b3800d667beee4d07b06d26530b356 in 37ms, sequenceid=91, compaction requested=true 2023-07-24 07:53:44,905 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:44,906 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:44,906 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:44,906 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:44,906 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,906 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:44,906 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:44,908 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,908 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,908 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,908 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 07:53:44,910 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32a5b052[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:44,910 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:44,910 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,911 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,911 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,911 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,912 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,912 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:44,912 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,912 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,912 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:44,912 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,912 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:44,912 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,912 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:44,912 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:44,912 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:44,913 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,913 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,913 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,914 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,915 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:44,918 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:44,918 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 07:53:44,918 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:44,918 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:44,918 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,918 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:44,920 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,920 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:44,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:44,920 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,920 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,920 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,920 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,921 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:44,920 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,921 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,921 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:44,923 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,923 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,924 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:44,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:44,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:44,929 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:44,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:44,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:44,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:44,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,929 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,930 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:44,930 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:44,930 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:44,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:44,930 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,930 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:44,930 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:44,930 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,930 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:44,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:44,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:45,030 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:45,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:45,039 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:53:45,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:53:45,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,041 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:45,041 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:45,041 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 07:53:45,041 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 07:53:45,041 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,041 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,042 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:45,043 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:45,043 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:45,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:45,042 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:45,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:45,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:45,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:45,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:45,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,043 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,043 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:53:45,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:45,044 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:45,044 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:45,044 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:45,045 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,044 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:53:45,045 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:45,045 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:53:45,045 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. started... 2023-07-24 07:53:45,045 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d0b3800d667beee4d07b06d26530b356 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-07-24 07:53:45,052 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,052 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,053 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,053 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:45,053 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,053 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:45,055 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/bf5cc82714314c92b1eb4e60558f6e6e 2023-07-24 07:53:45,059 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf5cc82714314c92b1eb4e60558f6e6e 2023-07-24 07:53:45,067 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/c1e23f94530942af9999f9881499d11d 2023-07-24 07:53:45,071 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e23f94530942af9999f9881499d11d 2023-07-24 07:53:45,072 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/bf5cc82714314c92b1eb4e60558f6e6e as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/bf5cc82714314c92b1eb4e60558f6e6e 2023-07-24 07:53:45,076 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf5cc82714314c92b1eb4e60558f6e6e 2023-07-24 07:53:45,077 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/bf5cc82714314c92b1eb4e60558f6e6e, entries=5, sequenceid=99, filesize=5.0 K 2023-07-24 07:53:45,077 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/c1e23f94530942af9999f9881499d11d as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/c1e23f94530942af9999f9881499d11d 2023-07-24 07:53:45,082 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e23f94530942af9999f9881499d11d 2023-07-24 07:53:45,082 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/c1e23f94530942af9999f9881499d11d, entries=5, sequenceid=99, filesize=5.0 K 2023-07-24 07:53:45,083 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for d0b3800d667beee4d07b06d26530b356 in 38ms, sequenceid=99, compaction requested=true 2023-07-24 07:53:45,083 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:45,084 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:45,084 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:53:45,084 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:53:45,084 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,084 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 07:53:45,084 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 07:53:45,085 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,085 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:45,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:45,086 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,086 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:53:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:45,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:45,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:45,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 07:53:45,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f31ed8c[Count = 0] remaining members to acquire global barrier 2023-07-24 07:53:45,088 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 07:53:45,088 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,089 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,089 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,089 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,089 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:45,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,089 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:45,089 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 07:53:45,089 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,089 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:53:45,089 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:45,089 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:53:45,089 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:45,089 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 07:53:45,090 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:53:45,090 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:53:45,091 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,091 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:45,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:45,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:45,091 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:45,091 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:45,093 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:45,093 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:45,093 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 07:53:45,093 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:45,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:45,093 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:45,093 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 07:53:45,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:45,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:45,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:45,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:45,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:45,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:45,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:53:45,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:53:45,096 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 07:53:45,096 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:53:45,096 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:53:45,096 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,096 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:53:45,097 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,097 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,097 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,097 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:45,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:53:45,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:53:45,097 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,097 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:45,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,097 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:53:45,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:45,098 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:45,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:45,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,098 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:53:45,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,099 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:45,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:45,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:45,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:53:45,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,100 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,101 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:45,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:45,101 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:45,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:45,103 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:45,103 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,103 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:45,103 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:45,103 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:45,103 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:53:45,103 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:45,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:45,103 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:45,103 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:53:45,103 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:45,104 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:45,103 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 07:53:45,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:53:45,103 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:53:45,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:53:45,104 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:45,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:45,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:53:45,104 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,104 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:53:45,104 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,104 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,104 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:45,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:45,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:45,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:45,104 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:53:45,105 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:53:45,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:45,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:53:45,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:53:45,205 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:53:45,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:53:45,208 DEBUG [Listener at localhost/42537] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,209 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:45,211 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:45,211 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:45,211 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:45,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d0b3800d667beee4d07b06d26530b356, UNASSIGN}] 2023-07-24 07:53:45,213 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d0b3800d667beee4d07b06d26530b356, UNASSIGN 2023-07-24 07:53:45,213 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=d0b3800d667beee4d07b06d26530b356, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,214 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690185225213"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185225213"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185225213"}]},"ts":"1690185225213"} 2023-07-24 07:53:45,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure d0b3800d667beee4d07b06d26530b356, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:45,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d0b3800d667beee4d07b06d26530b356, disabling compactions & flushes 2023-07-24 07:53:45,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:45,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:45,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. after waiting 0 ms 2023-07-24 07:53:45,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:45,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing d0b3800d667beee4d07b06d26530b356 2/2 column families, dataSize=50 B heapSize=720 B 2023-07-24 07:53:45,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/9ba21fa23cb3434d81bb0910b2bd319d 2023-07-24 07:53:45,382 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ba21fa23cb3434d81bb0910b2bd319d 2023-07-24 07:53:45,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/307345c9f8be42eea6c7c22e6d62a57b 2023-07-24 07:53:45,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 307345c9f8be42eea6c7c22e6d62a57b 2023-07-24 07:53:45,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf1/9ba21fa23cb3434d81bb0910b2bd319d as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9ba21fa23cb3434d81bb0910b2bd319d 2023-07-24 07:53:45,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ba21fa23cb3434d81bb0910b2bd319d 2023-07-24 07:53:45,400 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9ba21fa23cb3434d81bb0910b2bd319d, entries=1, sequenceid=103, filesize=4.9 K 2023-07-24 07:53:45,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.tmp/cf2/307345c9f8be42eea6c7c22e6d62a57b as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/307345c9f8be42eea6c7c22e6d62a57b 2023-07-24 07:53:45,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 307345c9f8be42eea6c7c22e6d62a57b 2023-07-24 07:53:45,405 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/307345c9f8be42eea6c7c22e6d62a57b, entries=1, sequenceid=103, filesize=4.9 K 2023-07-24 07:53:45,406 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for d0b3800d667beee4d07b06d26530b356 in 39ms, sequenceid=103, compaction requested=true 2023-07-24 07:53:45,411 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-07-24 07:53:45,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:45,412 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356. 2023-07-24 07:53:45,412 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d0b3800d667beee4d07b06d26530b356: 2023-07-24 07:53:45,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,414 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=d0b3800d667beee4d07b06d26530b356, regionState=CLOSED 2023-07-24 07:53:45,414 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690185225414"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185225414"}]},"ts":"1690185225414"} 2023-07-24 07:53:45,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-07-24 07:53:45,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure d0b3800d667beee4d07b06d26530b356, server=jenkins-hbase4.apache.org,46543,1690185199091 in 200 msec 2023-07-24 07:53:45,418 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-07-24 07:53:45,418 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d0b3800d667beee4d07b06d26530b356, UNASSIGN in 204 msec 2023-07-24 07:53:45,431 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=d0b3800d667beee4d07b06d26530b356, threads=16 2023-07-24 07:53:45,431 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,431 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,439 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/27d030625e774ccd81013befa296e9ea for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,439 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/307345c9f8be42eea6c7c22e6d62a57b for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,439 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/32de190c3385449084241048a0a316f3 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,439 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,440 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/4d847fa15e084eda8112bde0ac92a620 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,440 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/c1e23f94530942af9999f9881499d11d for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,442 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d089ce7056b64a2eb5344e62ae81be65 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,443 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,443 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,445 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 075b9f5882fc41758f6c8ed1e424dedd 2023-07-24 07:53:45,446 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2d2a3c58f2fe460daca46281e9d9cb9c for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,450 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/55efa58a337544738a9b478b5e65ff05 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,450 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,454 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9ba21fa23cb3434d81bb0910b2bd319d for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,455 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/a753d07b8e934a76b252c6ba04351524 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,457 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/32de190c3385449084241048a0a316f3, top=false 2023-07-24 07:53:45,461 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 307345c9f8be42eea6c7c22e6d62a57b 2023-07-24 07:53:45,463 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/55efa58a337544738a9b478b5e65ff05, top=false 2023-07-24 07:53:45,464 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/a753d07b8e934a76b252c6ba04351524, top=false 2023-07-24 07:53:45,465 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ba21fa23cb3434d81bb0910b2bd319d 2023-07-24 07:53:45,468 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ba21fa23cb3434d81bb0910b2bd319d 2023-07-24 07:53:45,468 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9ba21fa23cb3434d81bb0910b2bd319d, top=true 2023-07-24 07:53:45,468 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/4d847fa15e084eda8112bde0ac92a620, top=false 2023-07-24 07:53:45,469 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e23f94530942af9999f9881499d11d 2023-07-24 07:53:45,473 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27d030625e774ccd81013befa296e9ea 2023-07-24 07:53:45,473 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2d2a3c58f2fe460daca46281e9d9cb9c, top=false 2023-07-24 07:53:45,475 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9283b60dca114095a8dace3728aa08ac 2023-07-24 07:53:45,476 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d089ce7056b64a2eb5344e62ae81be65, top=false 2023-07-24 07:53:45,479 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e23f94530942af9999f9881499d11d 2023-07-24 07:53:45,479 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/c1e23f94530942af9999f9881499d11d, top=true 2023-07-24 07:53:45,480 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 307345c9f8be42eea6c7c22e6d62a57b 2023-07-24 07:53:45,480 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/307345c9f8be42eea6c7c22e6d62a57b, top=true 2023-07-24 07:53:45,481 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27d030625e774ccd81013befa296e9ea 2023-07-24 07:53:45,481 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/27d030625e774ccd81013befa296e9ea, top=true 2023-07-24 07:53:45,504 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-a753d07b8e934a76b252c6ba04351524 for child: bce71ec85c4b67fcb754f132e5380e54, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,505 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-32de190c3385449084241048a0a316f3 for child: bce71ec85c4b67fcb754f132e5380e54, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,507 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-4d847fa15e084eda8112bde0ac92a620 for child: bce71ec85c4b67fcb754f132e5380e54, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,508 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-d089ce7056b64a2eb5344e62ae81be65 for child: bce71ec85c4b67fcb754f132e5380e54, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,508 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-2d2a3c58f2fe460daca46281e9d9cb9c for child: bce71ec85c4b67fcb754f132e5380e54, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,512 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 075b9f5882fc41758f6c8ed1e424dedd 2023-07-24 07:53:45,512 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2d2a3c58f2fe460daca46281e9d9cb9c for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,512 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,513 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d089ce7056b64a2eb5344e62ae81be65 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,513 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/bf5cc82714314c92b1eb4e60558f6e6e for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,519 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-9ba21fa23cb3434d81bb0910b2bd319d for child: d31cb3be64e26e69f92b7e2238413984, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,519 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9ba21fa23cb3434d81bb0910b2bd319d for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,519 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/c63cb36288504968821e1cf4b9da9e51 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,520 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-307345c9f8be42eea6c7c22e6d62a57b for child: d31cb3be64e26e69f92b7e2238413984, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,520 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/307345c9f8be42eea6c7c22e6d62a57b for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,520 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,520 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-55efa58a337544738a9b478b5e65ff05 for child: bce71ec85c4b67fcb754f132e5380e54, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,542 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-27d030625e774ccd81013befa296e9ea for child: d31cb3be64e26e69f92b7e2238413984, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,542 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/27d030625e774ccd81013befa296e9ea for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,545 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c1e23f94530942af9999f9881499d11d for child: d31cb3be64e26e69f92b7e2238413984, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,545 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/c1e23f94530942af9999f9881499d11d for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,562 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9283b60dca114095a8dace3728aa08ac 2023-07-24 07:53:45,594 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,594 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/a753d07b8e934a76b252c6ba04351524 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,601 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,607 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/32de190c3385449084241048a0a316f3 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,607 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/4d847fa15e084eda8112bde0ac92a620 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,607 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf5cc82714314c92b1eb4e60558f6e6e 2023-07-24 07:53:45,608 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,608 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c63cb36288504968821e1cf4b9da9e51 2023-07-24 07:53:45,609 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/55efa58a337544738a9b478b5e65ff05 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,609 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,610 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,613 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf5cc82714314c92b1eb4e60558f6e6e 2023-07-24 07:53:45,614 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c63cb36288504968821e1cf4b9da9e51 2023-07-24 07:53:45,614 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/c63cb36288504968821e1cf4b9da9e51, top=true 2023-07-24 07:53:45,614 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/bf5cc82714314c92b1eb4e60558f6e6e, top=true 2023-07-24 07:53:45,622 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,622 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-bf5cc82714314c92b1eb4e60558f6e6e for child: d31cb3be64e26e69f92b7e2238413984, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,622 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/.splits/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c63cb36288504968821e1cf4b9da9e51 for child: d31cb3be64e26e69f92b7e2238413984, parent: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,622 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/bf5cc82714314c92b1eb4e60558f6e6e for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,622 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/c63cb36288504968821e1cf4b9da9e51 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,625 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49 for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,625 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f for region: d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:45,625 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region d0b3800d667beee4d07b06d26530b356 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 07:53:45,673 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-07-24 07:53:45,675 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-07-24 07:53:45,676 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690185225676"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1690185225676"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1690185225676"}]},"ts":"1690185225676"} 2023-07-24 07:53:45,677 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690185225676"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185225676"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185225676"}]},"ts":"1690185225676"} 2023-07-24 07:53:45,677 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690185225676"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185225676"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185225676"}]},"ts":"1690185225676"} 2023-07-24 07:53:45,680 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bce71ec85c4b67fcb754f132e5380e54, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d31cb3be64e26e69f92b7e2238413984, ASSIGN}] 2023-07-24 07:53:45,680 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d31cb3be64e26e69f92b7e2238413984, ASSIGN 2023-07-24 07:53:45,681 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d31cb3be64e26e69f92b7e2238413984, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:53:45,682 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bce71ec85c4b67fcb754f132e5380e54, ASSIGN 2023-07-24 07:53:45,682 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bce71ec85c4b67fcb754f132e5380e54, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:53:45,831 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 07:53:45,833 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=bce71ec85c4b67fcb754f132e5380e54, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,833 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690185225833"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185225833"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185225833"}]},"ts":"1690185225833"} 2023-07-24 07:53:45,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure bce71ec85c4b67fcb754f132e5380e54, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:45,837 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=d31cb3be64e26e69f92b7e2238413984, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:45,837 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690185225837"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185225837"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185225837"}]},"ts":"1690185225837"} 2023-07-24 07:53:45,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure d31cb3be64e26e69f92b7e2238413984, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:45,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:45,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bce71ec85c4b67fcb754f132e5380e54, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.', STARTKEY => '', ENDKEY => '60'} 2023-07-24 07:53:45,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:45,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:45,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:45,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:45,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:45,995 INFO [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:45,995 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1 2023-07-24 07:53:45,996 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1 2023-07-24 07:53:45,996 INFO [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bce71ec85c4b67fcb754f132e5380e54 columnFamilyName cf1 2023-07-24 07:53:46,006 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31-bottom 2023-07-24 07:53:46,010 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,011 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac-bottom 2023-07-24 07:53:46,016 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49-bottom 2023-07-24 07:53:46,021 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f-bottom 2023-07-24 07:53:46,025 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-2d2a3c58f2fe460daca46281e9d9cb9c 2023-07-24 07:53:46,029 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-55efa58a337544738a9b478b5e65ff05 2023-07-24 07:53:46,032 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-a753d07b8e934a76b252c6ba04351524 2023-07-24 07:53:46,033 INFO [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(310): Store=bce71ec85c4b67fcb754f132e5380e54/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:46,033 INFO [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:46,034 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2 2023-07-24 07:53:46,034 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2 2023-07-24 07:53:46,035 INFO [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bce71ec85c4b67fcb754f132e5380e54 columnFamilyName cf2 2023-07-24 07:53:46,046 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,048 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd-bottom 2023-07-24 07:53:46,054 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc-bottom 2023-07-24 07:53:46,059 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836-bottom 2023-07-24 07:53:46,063 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab-bottom 2023-07-24 07:53:46,067 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-32de190c3385449084241048a0a316f3 2023-07-24 07:53:46,071 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-4d847fa15e084eda8112bde0ac92a620 2023-07-24 07:53:46,074 DEBUG [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-d089ce7056b64a2eb5344e62ae81be65 2023-07-24 07:53:46,075 INFO [StoreOpener-bce71ec85c4b67fcb754f132e5380e54-1] regionserver.HStore(310): Store=bce71ec85c4b67fcb754f132e5380e54/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:46,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:46,077 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:46,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:46,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:46,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bce71ec85c4b67fcb754f132e5380e54; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67392087, jitterRate=0.004220351576805115}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:46,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bce71ec85c4b67fcb754f132e5380e54: 2023-07-24 07:53:46,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54., pid=77, masterSystemTime=1690185225986 2023-07-24 07:53:46,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:46,084 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:46,084 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:46,084 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 07:53:46,084 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,084 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1912): bce71ec85c4b67fcb754f132e5380e54/cf1 is initiating minor compaction (all files) 2023-07-24 07:53:46,084 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1912): bce71ec85c4b67fcb754f132e5380e54/cf2 is initiating minor compaction (all files) 2023-07-24 07:53:46,084 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of bce71ec85c4b67fcb754f132e5380e54/cf1 in testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,084 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2259): Starting compaction of bce71ec85c4b67fcb754f132e5380e54/cf2 in testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,084 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-a753d07b8e934a76b252c6ba04351524, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-2d2a3c58f2fe460daca46281e9d9cb9c, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-55efa58a337544738a9b478b5e65ff05, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac-bottom] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/.tmp, totalSize=35.0 K 2023-07-24 07:53:46,084 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-d089ce7056b64a2eb5344e62ae81be65, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-4d847fa15e084eda8112bde0ac92a620, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-32de190c3385449084241048a0a316f3, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd-bottom] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/.tmp, totalSize=35.0 K 2023-07-24 07:53:46,085 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185223528 2023-07-24 07:53:46,085 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185223528 2023-07-24 07:53:46,085 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185223715 2023-07-24 07:53:46,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d31cb3be64e26e69f92b7e2238413984, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.', STARTKEY => '60', ENDKEY => ''} 2023-07-24 07:53:46,086 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=bce71ec85c4b67fcb754f132e5380e54, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:46,086 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690185226086"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185226086"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185226086"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185226086"}]},"ts":"1690185226086"} 2023-07-24 07:53:46,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:46,087 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185223715 2023-07-24 07:53:46,087 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185223899 2023-07-24 07:53:46,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:46,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,087 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185223899 2023-07-24 07:53:46,087 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-a753d07b8e934a76b252c6ba04351524, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185224092 2023-07-24 07:53:46,088 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-d089ce7056b64a2eb5344e62ae81be65, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185224092 2023-07-24 07:53:46,088 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-2d2a3c58f2fe460daca46281e9d9cb9c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185224302 2023-07-24 07:53:46,088 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-55efa58a337544738a9b478b5e65ff05, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185224490 2023-07-24 07:53:46,089 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-4d847fa15e084eda8112bde0ac92a620, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185224302 2023-07-24 07:53:46,089 INFO [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,089 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690185224671 2023-07-24 07:53:46,089 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-32de190c3385449084241048a0a316f3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185224490 2023-07-24 07:53:46,090 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1 2023-07-24 07:53:46,090 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1 2023-07-24 07:53:46,090 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690185224671 2023-07-24 07:53:46,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-07-24 07:53:46,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure bce71ec85c4b67fcb754f132e5380e54, server=jenkins-hbase4.apache.org,46543,1690185199091 in 253 msec 2023-07-24 07:53:46,091 INFO [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d31cb3be64e26e69f92b7e2238413984 columnFamilyName cf1 2023-07-24 07:53:46,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bce71ec85c4b67fcb754f132e5380e54, ASSIGN in 411 msec 2023-07-24 07:53:46,106 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31-top 2023-07-24 07:53:46,109 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,110 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac-top 2023-07-24 07:53:46,115 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49-top 2023-07-24 07:53:46,119 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] throttle.PressureAwareThroughputController(145): bce71ec85c4b67fcb754f132e5380e54#cf1#compaction#72 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:46,124 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f-top 2023-07-24 07:53:46,125 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] throttle.PressureAwareThroughputController(145): bce71ec85c4b67fcb754f132e5380e54#cf2#compaction#73 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:46,127 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9ba21fa23cb3434d81bb0910b2bd319d 2023-07-24 07:53:46,127 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-9ba21fa23cb3434d81bb0910b2bd319d 2023-07-24 07:53:46,130 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bf5cc82714314c92b1eb4e60558f6e6e 2023-07-24 07:53:46,130 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-bf5cc82714314c92b1eb4e60558f6e6e 2023-07-24 07:53:46,137 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c63cb36288504968821e1cf4b9da9e51 2023-07-24 07:53:46,137 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c63cb36288504968821e1cf4b9da9e51 2023-07-24 07:53:46,140 INFO [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(310): Store=d31cb3be64e26e69f92b7e2238413984/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:46,140 INFO [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,141 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2 2023-07-24 07:53:46,141 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2 2023-07-24 07:53:46,141 INFO [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d31cb3be64e26e69f92b7e2238413984 columnFamilyName cf2 2023-07-24 07:53:46,156 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/.tmp/cf1/22e18520ad404d57866baae950d318aa as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/22e18520ad404d57866baae950d318aa 2023-07-24 07:53:46,157 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/.tmp/cf2/86842eafacb34117b23067c684595a6f as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/86842eafacb34117b23067c684595a6f 2023-07-24 07:53:46,162 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bce71ec85c4b67fcb754f132e5380e54/cf1 of bce71ec85c4b67fcb754f132e5380e54 into 22e18520ad404d57866baae950d318aa(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:46,162 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bce71ec85c4b67fcb754f132e5380e54/cf2 of bce71ec85c4b67fcb754f132e5380e54 into 86842eafacb34117b23067c684595a6f(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:46,162 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for bce71ec85c4b67fcb754f132e5380e54: 2023-07-24 07:53:46,162 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for bce71ec85c4b67fcb754f132e5380e54: 2023-07-24 07:53:46,162 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54., storeName=bce71ec85c4b67fcb754f132e5380e54/cf1, priority=9, startTime=1690185226082; duration=0sec 2023-07-24 07:53:46,162 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54., storeName=bce71ec85c4b67fcb754f132e5380e54/cf2, priority=9, startTime=1690185226084; duration=0sec 2023-07-24 07:53:46,162 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:46,162 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:46,164 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,164 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd-top 2023-07-24 07:53:46,167 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc-top 2023-07-24 07:53:46,171 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836-top 2023-07-24 07:53:46,176 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab-top 2023-07-24 07:53:46,182 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27d030625e774ccd81013befa296e9ea 2023-07-24 07:53:46,182 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-27d030625e774ccd81013befa296e9ea 2023-07-24 07:53:46,186 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 307345c9f8be42eea6c7c22e6d62a57b 2023-07-24 07:53:46,186 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-307345c9f8be42eea6c7c22e6d62a57b 2023-07-24 07:53:46,189 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1e23f94530942af9999f9881499d11d 2023-07-24 07:53:46,189 DEBUG [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c1e23f94530942af9999f9881499d11d 2023-07-24 07:53:46,190 INFO [StoreOpener-d31cb3be64e26e69f92b7e2238413984-1] regionserver.HStore(310): Store=d31cb3be64e26e69f92b7e2238413984/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:46,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:53:46,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d31cb3be64e26e69f92b7e2238413984; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64642821, jitterRate=-0.036746904253959656}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:53:46,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d31cb3be64e26e69f92b7e2238413984: 2023-07-24 07:53:46,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984., pid=78, masterSystemTime=1690185225986 2023-07-24 07:53:46,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:46,197 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:46,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:46,197 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:53:46,197 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,197 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1912): d31cb3be64e26e69f92b7e2238413984/cf1 is initiating minor compaction (all files) 2023-07-24 07:53:46,197 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,197 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1912): d31cb3be64e26e69f92b7e2238413984/cf2 is initiating minor compaction (all files) 2023-07-24 07:53:46,198 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d31cb3be64e26e69f92b7e2238413984/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,198 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2259): Starting compaction of d31cb3be64e26e69f92b7e2238413984/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,198 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-27d030625e774ccd81013befa296e9ea, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c1e23f94530942af9999f9881499d11d, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-307345c9f8be42eea6c7c22e6d62a57b] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/.tmp, totalSize=34.8 K 2023-07-24 07:53:46,198 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c63cb36288504968821e1cf4b9da9e51, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-bf5cc82714314c92b1eb4e60558f6e6e, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-9ba21fa23cb3434d81bb0910b2bd319d] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/.tmp, totalSize=34.8 K 2023-07-24 07:53:46,199 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690185223528 2023-07-24 07:53:46,199 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690185223528 2023-07-24 07:53:46,199 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690185223715 2023-07-24 07:53:46,199 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690185223715 2023-07-24 07:53:46,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,200 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690185223899 2023-07-24 07:53:46,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,200 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690185223899 2023-07-24 07:53:46,200 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1690185224671 2023-07-24 07:53:46,200 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=d31cb3be64e26e69f92b7e2238413984, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:46,201 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1690185224671 2023-07-24 07:53:46,201 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690185226200"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185226200"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185226200"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185226200"}]},"ts":"1690185226200"} 2023-07-24 07:53:46,201 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c63cb36288504968821e1cf4b9da9e51, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-07-24 07:53:46,202 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-27d030625e774ccd81013befa296e9ea, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-07-24 07:53:46,202 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-bf5cc82714314c92b1eb4e60558f6e6e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-07-24 07:53:46,202 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c1e23f94530942af9999f9881499d11d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-07-24 07:53:46,202 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-9ba21fa23cb3434d81bb0910b2bd319d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-24 07:53:46,203 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-307345c9f8be42eea6c7c22e6d62a57b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-24 07:53:46,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-07-24 07:53:46,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure d31cb3be64e26e69f92b7e2238413984, server=jenkins-hbase4.apache.org,46543,1690185199091 in 363 msec 2023-07-24 07:53:46,205 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 07:53:46,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-07-24 07:53:46,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d31cb3be64e26e69f92b7e2238413984, ASSIGN in 524 msec 2023-07-24 07:53:46,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 in 1.0000 sec 2023-07-24 07:53:46,223 DEBUG [Listener at localhost/42537] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 07:53:46,232 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35550, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 07:53:46,240 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,242 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-24 07:53:46,242 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.HStore(1912): bce71ec85c4b67fcb754f132e5380e54/cf1 is initiating major compaction (all files) 2023-07-24 07:53:46,242 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1f0a0aa9; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 07:53:46,242 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-24 07:53:46,242 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.HStore(1912): bce71ec85c4b67fcb754f132e5380e54/cf2 is initiating major compaction (all files) 2023-07-24 07:53:46,242 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@593a53f6; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-07-24 07:53:46,245 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] throttle.PressureAwareThroughputController(145): d31cb3be64e26e69f92b7e2238413984#cf1#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:46,245 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46543] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,245 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] throttle.PressureAwareThroughputController(145): d31cb3be64e26e69f92b7e2238413984#cf2#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:46,245 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46543] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-07-24 07:53:46,245 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46543] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-07-24 07:53:46,246 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46543] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-07-24 07:53:46,246 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46543] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. because compaction request was cancelled 2023-07-24 07:53:46,246 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46543] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-07-24 07:53:46,246 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46543] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-07-24 07:53:46,246 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46543] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-07-24 07:53:46,246 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46543] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. because compaction request was cancelled 2023-07-24 07:53:46,251 INFO [Listener at localhost/42537] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 07:53:46,273 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. is not mergeable because it has references 2023-07-24 07:53:46,274 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46543] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. is not mergeable because it has references 2023-07-24 07:53:46,279 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 65901227f822497a84f768de591ba282: NONE, but ROW specified in column family configuration 2023-07-24 07:53:46,280 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/.tmp/cf2/65901227f822497a84f768de591ba282 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/65901227f822497a84f768de591ba282 2023-07-24 07:53:46,283 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0afe9b1d233740ed80c56e276e095d56: NONE, but ROW specified in column family configuration 2023-07-24 07:53:46,284 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/.tmp/cf1/0afe9b1d233740ed80c56e276e095d56 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/0afe9b1d233740ed80c56e276e095d56 2023-07-24 07:53:46,285 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 65901227f822497a84f768de591ba282: NONE, but ROW specified in column family configuration 2023-07-24 07:53:46,286 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d31cb3be64e26e69f92b7e2238413984/cf2 of d31cb3be64e26e69f92b7e2238413984 into 65901227f822497a84f768de591ba282(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:46,286 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for d31cb3be64e26e69f92b7e2238413984: 2023-07-24 07:53:46,286 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984., storeName=d31cb3be64e26e69f92b7e2238413984/cf2, priority=9, startTime=1690185226197; duration=0sec 2023-07-24 07:53:46,286 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-07-24 07:53:46,286 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2259): Starting compaction of bce71ec85c4b67fcb754f132e5380e54/cf2 in testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,286 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/86842eafacb34117b23067c684595a6f] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/.tmp, totalSize=6.8 K 2023-07-24 07:53:46,287 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 86842eafacb34117b23067c684595a6f, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690185223528 2023-07-24 07:53:46,289 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 0afe9b1d233740ed80c56e276e095d56: NONE, but ROW specified in column family configuration 2023-07-24 07:53:46,290 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d31cb3be64e26e69f92b7e2238413984/cf1 of d31cb3be64e26e69f92b7e2238413984 into 0afe9b1d233740ed80c56e276e095d56(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:46,290 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d31cb3be64e26e69f92b7e2238413984: 2023-07-24 07:53:46,290 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984., storeName=d31cb3be64e26e69f92b7e2238413984/cf1, priority=9, startTime=1690185226196; duration=0sec 2023-07-24 07:53:46,290 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 07:53:46,290 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of bce71ec85c4b67fcb754f132e5380e54/cf1 in testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,290 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/22e18520ad404d57866baae950d318aa] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/.tmp, totalSize=6.8 K 2023-07-24 07:53:46,291 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 22e18520ad404d57866baae950d318aa, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690185223528 2023-07-24 07:53:46,291 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] throttle.PressureAwareThroughputController(145): bce71ec85c4b67fcb754f132e5380e54#cf2#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:46,295 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] throttle.PressureAwareThroughputController(145): bce71ec85c4b67fcb754f132e5380e54#cf1#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:53:46,306 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/.tmp/cf2/169d5c76ce7a46839440e0cdcf9af00b as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/169d5c76ce7a46839440e0cdcf9af00b 2023-07-24 07:53:46,309 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/.tmp/cf1/e880bae7f93946819d635f8c9d4272e5 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/e880bae7f93946819d635f8c9d4272e5 2023-07-24 07:53:46,311 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in bce71ec85c4b67fcb754f132e5380e54/cf2 of bce71ec85c4b67fcb754f132e5380e54 into 169d5c76ce7a46839440e0cdcf9af00b(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:46,311 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for bce71ec85c4b67fcb754f132e5380e54: 2023-07-24 07:53:46,313 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in bce71ec85c4b67fcb754f132e5380e54/cf1 of bce71ec85c4b67fcb754f132e5380e54 into e880bae7f93946819d635f8c9d4272e5(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:53:46,313 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for bce71ec85c4b67fcb754f132e5380e54: 2023-07-24 07:53:46,313 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1690185226242; duration=0sec 2023-07-24 07:53:46,313 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1690185226242; duration=0sec 2023-07-24 07:53:46,313 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:46,313 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:53:46,382 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. is not mergeable because it has references 2023-07-24 07:53:46,383 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46543] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. is not mergeable because it has references 2023-07-24 07:53:46,390 DEBUG [Listener at localhost/42537] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-07-24 07:53:46,392 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:46,394 INFO [Listener at localhost/42537] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:46,394 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:46,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:46,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-24 07:53:46,398 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185226398"}]},"ts":"1690185226398"} 2023-07-24 07:53:46,399 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-07-24 07:53:46,401 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-07-24 07:53:46,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bce71ec85c4b67fcb754f132e5380e54, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d31cb3be64e26e69f92b7e2238413984, UNASSIGN}] 2023-07-24 07:53:46,402 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d31cb3be64e26e69f92b7e2238413984, UNASSIGN 2023-07-24 07:53:46,403 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=d31cb3be64e26e69f92b7e2238413984, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:46,403 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690185226402"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185226402"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185226402"}]},"ts":"1690185226402"} 2023-07-24 07:53:46,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure d31cb3be64e26e69f92b7e2238413984, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:46,406 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bce71ec85c4b67fcb754f132e5380e54, UNASSIGN 2023-07-24 07:53:46,407 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=bce71ec85c4b67fcb754f132e5380e54, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:46,407 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690185226407"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185226407"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185226407"}]},"ts":"1690185226407"} 2023-07-24 07:53:46,408 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure bce71ec85c4b67fcb754f132e5380e54, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:46,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-24 07:53:46,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d31cb3be64e26e69f92b7e2238413984, disabling compactions & flushes 2023-07-24 07:53:46,556 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. after waiting 0 ms 2023-07-24 07:53:46,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,556 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c63cb36288504968821e1cf4b9da9e51, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-bf5cc82714314c92b1eb4e60558f6e6e, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-9ba21fa23cb3434d81bb0910b2bd319d] to archive 2023-07-24 07:53:46,557 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:46,559 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,560 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,561 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,563 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,564 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c63cb36288504968821e1cf4b9da9e51 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c63cb36288504968821e1cf4b9da9e51 2023-07-24 07:53:46,565 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-bf5cc82714314c92b1eb4e60558f6e6e to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-bf5cc82714314c92b1eb4e60558f6e6e 2023-07-24 07:53:46,566 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-9ba21fa23cb3434d81bb0910b2bd319d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-9ba21fa23cb3434d81bb0910b2bd319d 2023-07-24 07:53:46,567 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-27d030625e774ccd81013befa296e9ea, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c1e23f94530942af9999f9881499d11d, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-307345c9f8be42eea6c7c22e6d62a57b] to archive 2023-07-24 07:53:46,568 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:46,569 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,571 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,572 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,572 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,574 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-27d030625e774ccd81013befa296e9ea to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-27d030625e774ccd81013befa296e9ea 2023-07-24 07:53:46,575 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c1e23f94530942af9999f9881499d11d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-c1e23f94530942af9999f9881499d11d 2023-07-24 07:53:46,576 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-307345c9f8be42eea6c7c22e6d62a57b to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-307345c9f8be42eea6c7c22e6d62a57b 2023-07-24 07:53:46,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/recovered.edits/111.seqid, newMaxSeqId=111, maxSeqId=106 2023-07-24 07:53:46,580 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:46,581 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984. 2023-07-24 07:53:46,581 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d31cb3be64e26e69f92b7e2238413984: 2023-07-24 07:53:46,582 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,583 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:46,583 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bce71ec85c4b67fcb754f132e5380e54, disabling compactions & flushes 2023-07-24 07:53:46,583 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,583 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,583 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. after waiting 0 ms 2023-07-24 07:53:46,583 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,583 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=d31cb3be64e26e69f92b7e2238413984, regionState=CLOSED 2023-07-24 07:53:46,583 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690185226583"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185226583"}]},"ts":"1690185226583"} 2023-07-24 07:53:46,584 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-a753d07b8e934a76b252c6ba04351524, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-2d2a3c58f2fe460daca46281e9d9cb9c, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-55efa58a337544738a9b478b5e65ff05, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/22e18520ad404d57866baae950d318aa, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac-bottom] to archive 2023-07-24 07:53:46,585 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:46,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-07-24 07:53:46,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure d31cb3be64e26e69f92b7e2238413984, server=jenkins-hbase4.apache.org,46543,1690185199091 in 181 msec 2023-07-24 07:53:46,586 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/2858f8ef6d884085989a3a44e5c3bd31.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d31cb3be64e26e69f92b7e2238413984, UNASSIGN in 185 msec 2023-07-24 07:53:46,588 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/b33768ad46944051b4c5e938ec335c49.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,589 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/f501e4b5972443a9a6d6b6c94d95004f.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,590 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-a753d07b8e934a76b252c6ba04351524 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-a753d07b8e934a76b252c6ba04351524 2023-07-24 07:53:46,591 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-2d2a3c58f2fe460daca46281e9d9cb9c to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-2d2a3c58f2fe460daca46281e9d9cb9c 2023-07-24 07:53:46,592 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-55efa58a337544738a9b478b5e65ff05 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-55efa58a337544738a9b478b5e65ff05 2023-07-24 07:53:46,593 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/22e18520ad404d57866baae950d318aa to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/22e18520ad404d57866baae950d318aa 2023-07-24 07:53:46,594 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/9283b60dca114095a8dace3728aa08ac.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,596 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-d089ce7056b64a2eb5344e62ae81be65, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-4d847fa15e084eda8112bde0ac92a620, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-32de190c3385449084241048a0a316f3, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/86842eafacb34117b23067c684595a6f, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd-bottom] to archive 2023-07-24 07:53:46,597 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:53:46,598 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/10c4507131c5422e8dfbfd2ba643d5fc.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,599 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/3c41ef91fb684a13a915be1ef4405836.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,600 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/d863ceea01a347ff80130486a58ce7ab.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,601 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-d089ce7056b64a2eb5344e62ae81be65 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-d089ce7056b64a2eb5344e62ae81be65 2023-07-24 07:53:46,602 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-4d847fa15e084eda8112bde0ac92a620 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-4d847fa15e084eda8112bde0ac92a620 2023-07-24 07:53:46,603 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-32de190c3385449084241048a0a316f3 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/testSplitTableRegionDeletedRowsDaughter=d0b3800d667beee4d07b06d26530b356-32de190c3385449084241048a0a316f3 2023-07-24 07:53:46,604 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/86842eafacb34117b23067c684595a6f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/86842eafacb34117b23067c684595a6f 2023-07-24 07:53:46,605 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/075b9f5882fc41758f6c8ed1e424dedd.d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,611 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-07-24 07:53:46,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:46,613 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54. 2023-07-24 07:53:46,613 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bce71ec85c4b67fcb754f132e5380e54: 2023-07-24 07:53:46,615 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:46,615 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=bce71ec85c4b67fcb754f132e5380e54, regionState=CLOSED 2023-07-24 07:53:46,615 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690185226615"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185226615"}]},"ts":"1690185226615"} 2023-07-24 07:53:46,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-07-24 07:53:46,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure bce71ec85c4b67fcb754f132e5380e54, server=jenkins-hbase4.apache.org,46543,1690185199091 in 208 msec 2023-07-24 07:53:46,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-07-24 07:53:46,620 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=bce71ec85c4b67fcb754f132e5380e54, UNASSIGN in 217 msec 2023-07-24 07:53:46,620 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185226620"}]},"ts":"1690185226620"} 2023-07-24 07:53:46,621 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-07-24 07:53:46,623 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-07-24 07:53:46,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 230 msec 2023-07-24 07:53:46,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-24 07:53:46,700 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-07-24 07:53:46,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:46,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:46,703 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:46,704 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:46,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-07-24 07:53:46,709 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,709 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,709 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:46,711 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/recovered.edits] 2023-07-24 07:53:46,711 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/recovered.edits] 2023-07-24 07:53:46,711 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/recovered.edits] 2023-07-24 07:53:46,714 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/0afe9b1d233740ed80c56e276e095d56 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf1/0afe9b1d233740ed80c56e276e095d56 2023-07-24 07:53:46,714 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/e880bae7f93946819d635f8c9d4272e5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf1/e880bae7f93946819d635f8c9d4272e5 2023-07-24 07:53:46,714 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2858f8ef6d884085989a3a44e5c3bd31 2023-07-24 07:53:46,715 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2d2a3c58f2fe460daca46281e9d9cb9c to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/2d2a3c58f2fe460daca46281e9d9cb9c 2023-07-24 07:53:46,716 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/169d5c76ce7a46839440e0cdcf9af00b to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/cf2/169d5c76ce7a46839440e0cdcf9af00b 2023-07-24 07:53:46,716 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/65901227f822497a84f768de591ba282 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/cf2/65901227f822497a84f768de591ba282 2023-07-24 07:53:46,717 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/55efa58a337544738a9b478b5e65ff05 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/55efa58a337544738a9b478b5e65ff05 2023-07-24 07:53:46,718 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9283b60dca114095a8dace3728aa08ac 2023-07-24 07:53:46,719 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/recovered.edits/113.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54/recovered.edits/113.seqid 2023-07-24 07:53:46,720 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/recovered.edits/111.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984/recovered.edits/111.seqid 2023-07-24 07:53:46,720 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9ba21fa23cb3434d81bb0910b2bd319d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/9ba21fa23cb3434d81bb0910b2bd319d 2023-07-24 07:53:46,720 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/bce71ec85c4b67fcb754f132e5380e54 2023-07-24 07:53:46,720 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:53:46,721 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/a753d07b8e934a76b252c6ba04351524 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/a753d07b8e934a76b252c6ba04351524 2023-07-24 07:53:46,722 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/b33768ad46944051b4c5e938ec335c49 2023-07-24 07:53:46,723 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/bf5cc82714314c92b1eb4e60558f6e6e to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/bf5cc82714314c92b1eb4e60558f6e6e 2023-07-24 07:53:46,724 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/c63cb36288504968821e1cf4b9da9e51 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/c63cb36288504968821e1cf4b9da9e51 2023-07-24 07:53:46,726 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf1/f501e4b5972443a9a6d6b6c94d95004f 2023-07-24 07:53:46,728 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/075b9f5882fc41758f6c8ed1e424dedd 2023-07-24 07:53:46,729 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/10c4507131c5422e8dfbfd2ba643d5fc 2023-07-24 07:53:46,730 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/27d030625e774ccd81013befa296e9ea to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/27d030625e774ccd81013befa296e9ea 2023-07-24 07:53:46,731 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/307345c9f8be42eea6c7c22e6d62a57b to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/307345c9f8be42eea6c7c22e6d62a57b 2023-07-24 07:53:46,732 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/32de190c3385449084241048a0a316f3 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/32de190c3385449084241048a0a316f3 2023-07-24 07:53:46,733 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/3c41ef91fb684a13a915be1ef4405836 2023-07-24 07:53:46,734 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/4d847fa15e084eda8112bde0ac92a620 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/4d847fa15e084eda8112bde0ac92a620 2023-07-24 07:53:46,735 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/c1e23f94530942af9999f9881499d11d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/c1e23f94530942af9999f9881499d11d 2023-07-24 07:53:46,737 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d089ce7056b64a2eb5344e62ae81be65 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d089ce7056b64a2eb5344e62ae81be65 2023-07-24 07:53:46,738 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/cf2/d863ceea01a347ff80130486a58ce7ab 2023-07-24 07:53:46,740 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/recovered.edits/106.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356/recovered.edits/106.seqid 2023-07-24 07:53:46,741 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d0b3800d667beee4d07b06d26530b356 2023-07-24 07:53:46,741 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-07-24 07:53:46,744 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:46,746 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-07-24 07:53:46,748 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-07-24 07:53:46,749 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:46,749 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-07-24 07:53:46,749 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185226749"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:46,749 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185226749"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:46,750 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185226749"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:46,751 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 07:53:46,752 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d0b3800d667beee4d07b06d26530b356, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1690185223023.d0b3800d667beee4d07b06d26530b356.', STARTKEY => '', ENDKEY => ''}, {ENCODED => bce71ec85c4b67fcb754f132e5380e54, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1690185225208.bce71ec85c4b67fcb754f132e5380e54.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => d31cb3be64e26e69f92b7e2238413984, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1690185225208.d31cb3be64e26e69f92b7e2238413984.', STARTKEY => '60', ENDKEY => ''}] 2023-07-24 07:53:46,752 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-07-24 07:53:46,752 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690185226752"}]},"ts":"9223372036854775807"} 2023-07-24 07:53:46,753 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-07-24 07:53:46,755 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:53:46,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 54 msec 2023-07-24 07:53:46,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-07-24 07:53:46,806 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-07-24 07:53:46,824 INFO [Listener at localhost/42537] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=414 (was 409) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:39240 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:49106 [Waiting for operation #54] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:48998 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:48982 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:49014 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:39170 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:49306 [Waiting for operation #52] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:59280 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:39200 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:51926 [Waiting for operation #69] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:59266 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:59828 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:59548 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:49028 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:39490 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1157322523_17 at /127.0.0.1:48976 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x58a9b8f-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=864 (was 862) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=300 (was 318), ProcessCount=174 (was 174), AvailableMemoryMB=8908 (was 8917) 2023-07-24 07:53:46,839 INFO [Listener at localhost/42537] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=414, OpenFileDescriptor=864, MaxFileDescriptor=60000, SystemLoadAverage=300, ProcessCount=174, AvailableMemoryMB=8907 2023-07-24 07:53:46,840 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:53:46,842 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 07:53:46,842 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 07:53:46,843 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:53:46,845 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 07:53:46,846 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 07:53:46,847 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:46,848 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d empty. 2023-07-24 07:53:46,848 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:46,848 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-07-24 07:53:46,858 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-07-24 07:53:46,859 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => 0be6421af829d9242b43814b95c3339d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp 2023-07-24 07:53:46,868 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:46,868 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing 0be6421af829d9242b43814b95c3339d, disabling compactions & flushes 2023-07-24 07:53:46,868 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:46,868 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:46,868 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. after waiting 0 ms 2023-07-24 07:53:46,868 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:46,868 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:46,868 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for 0be6421af829d9242b43814b95c3339d: 2023-07-24 07:53:46,870 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 07:53:46,871 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185226871"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185226871"}]},"ts":"1690185226871"} 2023-07-24 07:53:46,872 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 07:53:46,873 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 07:53:46,874 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185226873"}]},"ts":"1690185226873"} 2023-07-24 07:53:46,875 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-07-24 07:53:46,879 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:53:46,880 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:53:46,880 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:53:46,880 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:53:46,880 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:53:46,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0be6421af829d9242b43814b95c3339d, ASSIGN}] 2023-07-24 07:53:46,881 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0be6421af829d9242b43814b95c3339d, ASSIGN 2023-07-24 07:53:46,882 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0be6421af829d9242b43814b95c3339d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37809,1690185199136; forceNewPlan=false, retain=false 2023-07-24 07:53:47,032 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:53:47,035 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=0be6421af829d9242b43814b95c3339d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:47,035 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185227035"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185227035"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185227035"}]},"ts":"1690185227035"} 2023-07-24 07:53:47,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 0be6421af829d9242b43814b95c3339d, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:53:47,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:47,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0be6421af829d9242b43814b95c3339d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:53:47,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:47,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:47,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,196 INFO [StoreOpener-0be6421af829d9242b43814b95c3339d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,197 DEBUG [StoreOpener-0be6421af829d9242b43814b95c3339d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/cf1 2023-07-24 07:53:47,197 DEBUG [StoreOpener-0be6421af829d9242b43814b95c3339d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/cf1 2023-07-24 07:53:47,197 INFO [StoreOpener-0be6421af829d9242b43814b95c3339d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0be6421af829d9242b43814b95c3339d columnFamilyName cf1 2023-07-24 07:53:47,198 INFO [StoreOpener-0be6421af829d9242b43814b95c3339d-1] regionserver.HStore(310): Store=0be6421af829d9242b43814b95c3339d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:47,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:53:47,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0be6421af829d9242b43814b95c3339d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68386172, jitterRate=0.019033372402191162}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 07:53:47,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0be6421af829d9242b43814b95c3339d: 2023-07-24 07:53:47,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d., pid=87, masterSystemTime=1690185227188 2023-07-24 07:53:47,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:47,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:47,206 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=0be6421af829d9242b43814b95c3339d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:47,206 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185227206"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185227206"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185227206"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185227206"}]},"ts":"1690185227206"} 2023-07-24 07:53:47,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-07-24 07:53:47,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 0be6421af829d9242b43814b95c3339d, server=jenkins-hbase4.apache.org,37809,1690185199136 in 171 msec 2023-07-24 07:53:47,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-07-24 07:53:47,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0be6421af829d9242b43814b95c3339d, ASSIGN in 329 msec 2023-07-24 07:53:47,211 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 07:53:47,211 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185227211"}]},"ts":"1690185227211"} 2023-07-24 07:53:47,212 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-07-24 07:53:47,221 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 07:53:47,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 379 msec 2023-07-24 07:53:47,352 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 07:53:47,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:53:47,377 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-07-24 07:53:47,378 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-07-24 07:53:47,378 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-07-24 07:53:47,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-07-24 07:53:47,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0be6421af829d9242b43814b95c3339d, REOPEN/MOVE}] 2023-07-24 07:53:47,394 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0be6421af829d9242b43814b95c3339d, REOPEN/MOVE 2023-07-24 07:53:47,395 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0be6421af829d9242b43814b95c3339d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:47,395 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185227395"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185227395"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185227395"}]},"ts":"1690185227395"} 2023-07-24 07:53:47,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 0be6421af829d9242b43814b95c3339d, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:53:47,548 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0be6421af829d9242b43814b95c3339d, disabling compactions & flushes 2023-07-24 07:53:47,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:47,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:47,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. after waiting 0 ms 2023-07-24 07:53:47,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:47,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 07:53:47,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:53:47,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:47,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0be6421af829d9242b43814b95c3339d: 2023-07-24 07:53:47,553 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 0be6421af829d9242b43814b95c3339d to self. 2023-07-24 07:53:47,555 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,555 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0be6421af829d9242b43814b95c3339d, regionState=CLOSED 2023-07-24 07:53:47,555 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185227555"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185227555"}]},"ts":"1690185227555"} 2023-07-24 07:53:47,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-07-24 07:53:47,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 0be6421af829d9242b43814b95c3339d, server=jenkins-hbase4.apache.org,37809,1690185199136 in 160 msec 2023-07-24 07:53:47,558 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0be6421af829d9242b43814b95c3339d, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37809,1690185199136; forceNewPlan=false, retain=true 2023-07-24 07:53:47,708 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:53:47,709 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0be6421af829d9242b43814b95c3339d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:47,709 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185227709"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185227709"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185227709"}]},"ts":"1690185227709"} 2023-07-24 07:53:47,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 0be6421af829d9242b43814b95c3339d, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:53:47,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:47,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0be6421af829d9242b43814b95c3339d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:53:47,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:47,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:47,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,867 INFO [StoreOpener-0be6421af829d9242b43814b95c3339d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,867 DEBUG [StoreOpener-0be6421af829d9242b43814b95c3339d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/cf1 2023-07-24 07:53:47,867 DEBUG [StoreOpener-0be6421af829d9242b43814b95c3339d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/cf1 2023-07-24 07:53:47,868 INFO [StoreOpener-0be6421af829d9242b43814b95c3339d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0be6421af829d9242b43814b95c3339d columnFamilyName cf1 2023-07-24 07:53:47,868 INFO [StoreOpener-0be6421af829d9242b43814b95c3339d-1] regionserver.HStore(310): Store=0be6421af829d9242b43814b95c3339d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:53:47,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0be6421af829d9242b43814b95c3339d 2023-07-24 07:53:47,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0be6421af829d9242b43814b95c3339d; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62199126, jitterRate=-0.0731607973575592}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 07:53:47,872 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0be6421af829d9242b43814b95c3339d: 2023-07-24 07:53:47,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d., pid=92, masterSystemTime=1690185227862 2023-07-24 07:53:47,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:47,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:53:47,875 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0be6421af829d9242b43814b95c3339d, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:53:47,875 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185227875"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185227875"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185227875"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185227875"}]},"ts":"1690185227875"} 2023-07-24 07:53:47,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-07-24 07:53:47,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 0be6421af829d9242b43814b95c3339d, server=jenkins-hbase4.apache.org,37809,1690185199136 in 165 msec 2023-07-24 07:53:47,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-07-24 07:53:47,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0be6421af829d9242b43814b95c3339d, REOPEN/MOVE in 485 msec 2023-07-24 07:53:47,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-07-24 07:53:47,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 501 msec 2023-07-24 07:53:47,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50, ASSIGN}] 2023-07-24 07:53:47,886 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50, ASSIGN 2023-07-24 07:53:47,886 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-24 07:53:48,037 DEBUG [jenkins-hbase4:37123] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:53:48,038 DEBUG [jenkins-hbase4:37123] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:53:48,038 DEBUG [jenkins-hbase4:37123] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:53:48,038 DEBUG [jenkins-hbase4:37123] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:53:48,038 DEBUG [jenkins-hbase4:37123] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-07-24 07:53:48,039 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=a0ab33d4ecb05d862ed118a501d92b50, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:53:48,040 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185228039"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1690185228039"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1690185228039"}]},"ts":"1690185228039"} 2023-07-24 07:53:48,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure a0ab33d4ecb05d862ed118a501d92b50, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:53:48,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50. 2023-07-24 07:53:48,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a0ab33d4ecb05d862ed118a501d92b50, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-07-24 07:53:48,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:53:48,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica a0ab33d4ecb05d862ed118a501d92b50 2023-07-24 07:53:48,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:53:48,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a0ab33d4ecb05d862ed118a501d92b50 2023-07-24 07:53:48,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a0ab33d4ecb05d862ed118a501d92b50 2023-07-24 07:53:48,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:48,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:48,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:48,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:48,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:48,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:48,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:48,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:49,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:49,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:49,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:49,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:49,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:49,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:49,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:49,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:49,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:49,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:50,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:50,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:50,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:50,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:50,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:50,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:50,569 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-07-24 07:53:50,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:50,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:50,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:50,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:51,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:51,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:51,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:51,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:51,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:51,492 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 07:53:51,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:51,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:51,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:51,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:51,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:52,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:52,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:52,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:52,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:52,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:52,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:52,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:52,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:52,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:52,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:53,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:53,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:53,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:53,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:53,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:53,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:53,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:53,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:53,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:53,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:54,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:54,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:54,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:54,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:54,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:54,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:54,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:54,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:54,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:54,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:55,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:55,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:55,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:55,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:55,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:55,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:55,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:55,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:55,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:55,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:56,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:56,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:56,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:56,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:56,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:56,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:56,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:56,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:56,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:56,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:57,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:57,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:57,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:57,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:57,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:57,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:57,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:57,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:57,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:57,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:58,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:58,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:58,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:58,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:58,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:58,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:58,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:58,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:58,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:58,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:59,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:59,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:59,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:59,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:59,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:59,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:59,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:59,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:59,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:53:59,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:00,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:00,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:00,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:00,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:00,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:00,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:00,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:00,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:00,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:00,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:01,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:01,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:01,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:01,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:01,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:01,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:01,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:01,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:01,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:01,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:02,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:02,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:02,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:02,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:02,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:02,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:02,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:02,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:02,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:02,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:03,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:03,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:03,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:03,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:03,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:03,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:03,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:03,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:03,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:03,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:04,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:04,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:04,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:04,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:04,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:04,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:04,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:04,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:04,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:04,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:05,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:05,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:05,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:05,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:05,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:05,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:05,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:05,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:05,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:05,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:06,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:06,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:06,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:06,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:06,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:06,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:06,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:06,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:06,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:06,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:07,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:07,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:07,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:07,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:07,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:07,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:07,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:07,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:07,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:07,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:08,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:08,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:08,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:08,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:08,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:08,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:08,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:08,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:08,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:08,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:09,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:09,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:09,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:09,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:09,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:09,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:09,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:09,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:09,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:09,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:10,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:10,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:10,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:10,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:10,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:10,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:10,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:10,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:10,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:10,947 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:11,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:11,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:11,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:11,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:11,448 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:11,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:11,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:11,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:11,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:11,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:12,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:12,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:12,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:12,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:12,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:12,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:12,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:12,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:12,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:12,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:13,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:13,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:13,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:13,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:13,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:13,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:13,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:13,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:13,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:13,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:14,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:14,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:14,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:14,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:14,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:14,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:14,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:14,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:14,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:14,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:15,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:15,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:15,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:15,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:15,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:15,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:15,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:15,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:15,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:15,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:16,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:16,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:16,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:16,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:16,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:16,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:16,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:16,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:16,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:16,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:17,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:17,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:17,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:17,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:17,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:17,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:17,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:17,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:17,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:17,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:18,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:18,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:18,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:18,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:18,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:18,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:18,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:18,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:18,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:18,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:19,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:19,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:19,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:19,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:19,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:19,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:19,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:19,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:19,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:19,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:20,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:20,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:20,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:20,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:20,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:20,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:20,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:20,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:20,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:20,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:21,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:21,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:21,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:21,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:21,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:21,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:21,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:21,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:21,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:21,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:22,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:22,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:22,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:22,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:22,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:22,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:22,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:22,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:22,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:22,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:23,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:23,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:23,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:23,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:23,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:23,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:23,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:23,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:23,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:23,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:24,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:24,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:24,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:24,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:24,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:24,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:24,677 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:24,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:24,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:24,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:25,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:25,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:25,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:25,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:25,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:25,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:25,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:25,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:25,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:25,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:26,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:26,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:26,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:26,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:26,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:26,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:26,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:26,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:26,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:26,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:27,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:27,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:27,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:27,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:27,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:27,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:27,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:27,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:27,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:27,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:28,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:28,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:28,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:28,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:28,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:28,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:28,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:28,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:28,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:28,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:29,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:29,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:29,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:29,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:29,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:29,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:29,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:29,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:29,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:29,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:30,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:30,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:30,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:30,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:30,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:30,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:30,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:30,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:30,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:30,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:31,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:31,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:31,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:31,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:31,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:31,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:31,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:31,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:31,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:31,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:32,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:32,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:32,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:32,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:32,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:32,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:32,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:32,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:32,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:32,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:33,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:33,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:33,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:33,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:33,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:33,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:33,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:33,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:33,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:33,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:34,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:34,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:34,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:34,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:34,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:34,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:34,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:34,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:34,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:34,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:35,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:35,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:35,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:35,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:35,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:35,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:35,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:35,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:35,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:36,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:36,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:36,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:36,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:36,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:36,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:36,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:36,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:36,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:36,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:37,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:37,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:37,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:37,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:37,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:37,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:37,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:37,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:37,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:37,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:38,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:38,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:38,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:38,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:38,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:38,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:38,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:38,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:38,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:38,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:39,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:39,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:39,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:39,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:39,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:39,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:39,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:39,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:39,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:39,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:40,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:40,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:40,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:40,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:40,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:40,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:40,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:40,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:40,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:40,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:41,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:41,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:41,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:41,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:41,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:41,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:41,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:41,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:41,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:41,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:42,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:42,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:42,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:42,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:42,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:42,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:42,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:42,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:42,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:42,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:43,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:43,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:43,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:43,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:43,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:43,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:43,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:43,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:43,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:43,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:44,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:44,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:44,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:44,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:44,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:44,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:44,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:44,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:44,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:44,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:45,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:45,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:45,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:45,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:45,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:45,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:45,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:45,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:45,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:45,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:46,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:46,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:46,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:46,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:46,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:46,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:46,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:46,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:46,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:46,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:47,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:47,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:47,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:47,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:47,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:47,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 07:54:47,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:47,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 07:54:47,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:47,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:47,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 07:54:47,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:47,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:48,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:48,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 07:54:48,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:48,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:48,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:48,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:48,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:48,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 07:54:48,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:48,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:48,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:48,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:49,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:49,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:49,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:49,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:49,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:49,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:49,579 DEBUG [Listener at localhost/42537] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:49,580 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:49,581 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:49,581 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:49,581 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:49,583 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 0be6421af829d9242b43814b95c3339d, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:49,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c exec-time=6 msec 2023-07-24 07:54:49,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:49,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 07:54:49,831 INFO [StoreOpener-a0ab33d4ecb05d862ed118a501d92b50-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a0ab33d4ecb05d862ed118a501d92b50 2023-07-24 07:54:49,835 DEBUG [StoreOpener-a0ab33d4ecb05d862ed118a501d92b50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/cf1 2023-07-24 07:54:49,836 DEBUG [StoreOpener-a0ab33d4ecb05d862ed118a501d92b50-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/cf1 2023-07-24 07:54:49,836 INFO [StoreOpener-a0ab33d4ecb05d862ed118a501d92b50-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a0ab33d4ecb05d862ed118a501d92b50 columnFamilyName cf1 2023-07-24 07:54:49,837 INFO [StoreOpener-a0ab33d4ecb05d862ed118a501d92b50-1] regionserver.HStore(310): Store=a0ab33d4ecb05d862ed118a501d92b50/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:54:49,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a0ab33d4ecb05d862ed118a501d92b50; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73786575, jitterRate=0.09950564801692963}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 07:54:49,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a0ab33d4ecb05d862ed118a501d92b50: 2023-07-24 07:54:49,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50., pid=94, masterSystemTime=1690185228195 2023-07-24 07:54:49,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50. 2023-07-24 07:54:49,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50. 2023-07-24 07:54:49,841 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=a0ab33d4ecb05d862ed118a501d92b50, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:49,841 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185289841"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1690185289841"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1690185289841"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1690185289841"}]},"ts":"1690185289841"} 2023-07-24 07:54:49,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-07-24 07:54:49,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure a0ab33d4ecb05d862ed118a501d92b50, server=jenkins-hbase4.apache.org,46543,1690185199091 in 1 mins, 1.801 sec 2023-07-24 07:54:49,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-07-24 07:54:49,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50, ASSIGN in 1 mins, 1.96 sec 2023-07-24 07:54:49,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.491 sec 2023-07-24 07:54:49,848 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1690185227350","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62498,"client":"172.31.14.131:54160","queuetimems":0,"class":"HMaster"} 2023-07-24 07:54:49,849 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-07-24 07:54:51,830 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up 2023-07-24 07:54:51,832 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:54:51,834 INFO [Listener at localhost/42537] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:51,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:51,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:51,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-24 07:54:51,838 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185291838"}]},"ts":"1690185291838"} 2023-07-24 07:54:51,839 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-07-24 07:54:51,840 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-07-24 07:54:51,841 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0be6421af829d9242b43814b95c3339d, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50, UNASSIGN}] 2023-07-24 07:54:51,842 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50, UNASSIGN 2023-07-24 07:54:51,842 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=a0ab33d4ecb05d862ed118a501d92b50, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:51,842 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185291842"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1690185291842"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1690185291842"}]},"ts":"1690185291842"} 2023-07-24 07:54:51,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure a0ab33d4ecb05d862ed118a501d92b50, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:54:51,845 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0be6421af829d9242b43814b95c3339d, UNASSIGN 2023-07-24 07:54:51,846 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=0be6421af829d9242b43814b95c3339d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:51,846 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185291846"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185291846"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185291846"}]},"ts":"1690185291846"} 2023-07-24 07:54:51,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 0be6421af829d9242b43814b95c3339d, server=jenkins-hbase4.apache.org,37809,1690185199136}] 2023-07-24 07:54:51,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-24 07:54:51,996 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a0ab33d4ecb05d862ed118a501d92b50 2023-07-24 07:54:51,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a0ab33d4ecb05d862ed118a501d92b50, disabling compactions & flushes 2023-07-24 07:54:51,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50. 2023-07-24 07:54:51,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50. 2023-07-24 07:54:51,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50. after waiting 0 ms 2023-07-24 07:54:51,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50. 2023-07-24 07:54:51,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:54:51,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50. 2023-07-24 07:54:51,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a0ab33d4ecb05d862ed118a501d92b50: 2023-07-24 07:54:51,999 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0be6421af829d9242b43814b95c3339d 2023-07-24 07:54:51,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0be6421af829d9242b43814b95c3339d, disabling compactions & flushes 2023-07-24 07:54:51,999 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:54:51,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:54:51,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. after waiting 0 ms 2023-07-24 07:54:51,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:54:52,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a0ab33d4ecb05d862ed118a501d92b50 2023-07-24 07:54:52,001 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=a0ab33d4ecb05d862ed118a501d92b50, regionState=CLOSED 2023-07-24 07:54:52,002 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185292000"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1690185292000"}]},"ts":"1690185292000"} 2023-07-24 07:54:52,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-24 07:54:52,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-07-24 07:54:52,006 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure a0ab33d4ecb05d862ed118a501d92b50, server=jenkins-hbase4.apache.org,46543,1690185199091 in 161 msec 2023-07-24 07:54:52,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:54:52,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d. 2023-07-24 07:54:52,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0be6421af829d9242b43814b95c3339d: 2023-07-24 07:54:52,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50, UNASSIGN in 165 msec 2023-07-24 07:54:52,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0be6421af829d9242b43814b95c3339d 2023-07-24 07:54:52,009 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=0be6421af829d9242b43814b95c3339d, regionState=CLOSED 2023-07-24 07:54:52,009 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690185292009"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185292009"}]},"ts":"1690185292009"} 2023-07-24 07:54:52,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-07-24 07:54:52,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 0be6421af829d9242b43814b95c3339d, server=jenkins-hbase4.apache.org,37809,1690185199136 in 163 msec 2023-07-24 07:54:52,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-07-24 07:54:52,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0be6421af829d9242b43814b95c3339d, UNASSIGN in 170 msec 2023-07-24 07:54:52,013 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185292013"}]},"ts":"1690185292013"} 2023-07-24 07:54:52,014 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-07-24 07:54:52,016 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-07-24 07:54:52,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 182 msec 2023-07-24 07:54:52,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-24 07:54:52,140 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-07-24 07:54:52,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:52,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:52,143 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:52,144 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:52,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-24 07:54:52,149 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d 2023-07-24 07:54:52,151 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/recovered.edits] 2023-07-24 07:54:52,156 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/recovered.edits/7.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d/recovered.edits/7.seqid 2023-07-24 07:54:52,157 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0be6421af829d9242b43814b95c3339d 2023-07-24 07:54:52,157 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-07-24 07:54:52,159 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:52,162 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-07-24 07:54:52,164 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-07-24 07:54:52,165 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:52,165 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-07-24 07:54:52,165 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185292165"}]},"ts":"9223372036854775807"} 2023-07-24 07:54:52,165 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185292165"}]},"ts":"9223372036854775807"} 2023-07-24 07:54:52,167 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-07-24 07:54:52,167 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0be6421af829d9242b43814b95c3339d, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690185226843.0be6421af829d9242b43814b95c3339d.', STARTKEY => '', ENDKEY => ''}, {ENCODED => a0ab33d4ecb05d862ed118a501d92b50, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690185226843_0001.a0ab33d4ecb05d862ed118a501d92b50.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-07-24 07:54:52,167 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-07-24 07:54:52,167 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690185292167"}]},"ts":"9223372036854775807"} 2023-07-24 07:54:52,171 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-07-24 07:54:52,173 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:52,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 32 msec 2023-07-24 07:54:52,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-24 07:54:52,246 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-07-24 07:54:52,262 INFO [Listener at localhost/42537] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=350 (was 414), OpenFileDescriptor=659 (was 864), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=168 (was 300), ProcessCount=171 (was 174), AvailableMemoryMB=8902 (was 8907) 2023-07-24 07:54:52,276 INFO [Listener at localhost/42537] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=351, OpenFileDescriptor=659, MaxFileDescriptor=60000, SystemLoadAverage=168, ProcessCount=171, AvailableMemoryMB=8902 2023-07-24 07:54:52,276 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:54:52,279 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 07:54:52,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 07:54:52,281 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:52,282 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 07:54:52,283 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 07:54:52,285 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:52,285 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29 empty. 2023-07-24 07:54:52,285 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:52,286 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-07-24 07:54:52,302 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-07-24 07:54:52,303 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => bd7787ba2e893a0cc2703ffc5a63bf29, NAME => 'testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp 2023-07-24 07:54:52,311 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:54:52,311 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing bd7787ba2e893a0cc2703ffc5a63bf29, disabling compactions & flushes 2023-07-24 07:54:52,311 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:52,311 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:52,311 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. after waiting 0 ms 2023-07-24 07:54:52,311 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:52,311 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:52,311 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for bd7787ba2e893a0cc2703ffc5a63bf29: 2023-07-24 07:54:52,313 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 07:54:52,314 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1690185292314"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185292314"}]},"ts":"1690185292314"} 2023-07-24 07:54:52,316 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 07:54:52,316 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 07:54:52,317 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185292316"}]},"ts":"1690185292316"} 2023-07-24 07:54:52,318 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-07-24 07:54:52,320 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:54:52,321 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:54:52,321 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:54:52,321 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:54:52,321 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:54:52,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=bd7787ba2e893a0cc2703ffc5a63bf29, ASSIGN}] 2023-07-24 07:54:52,323 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=bd7787ba2e893a0cc2703ffc5a63bf29, ASSIGN 2023-07-24 07:54:52,324 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=bd7787ba2e893a0cc2703ffc5a63bf29, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:54:52,474 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:54:52,475 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=bd7787ba2e893a0cc2703ffc5a63bf29, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,475 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1690185292475"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185292475"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185292475"}]},"ts":"1690185292475"} 2023-07-24 07:54:52,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure bd7787ba2e893a0cc2703ffc5a63bf29, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:54:52,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:52,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bd7787ba2e893a0cc2703ffc5a63bf29, NAME => 'testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:54:52,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:54:52,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:52,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:54:52,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:52,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:52,633 INFO [StoreOpener-bd7787ba2e893a0cc2703ffc5a63bf29-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:52,634 DEBUG [StoreOpener-bd7787ba2e893a0cc2703ffc5a63bf29-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1 2023-07-24 07:54:52,635 DEBUG [StoreOpener-bd7787ba2e893a0cc2703ffc5a63bf29-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1 2023-07-24 07:54:52,635 INFO [StoreOpener-bd7787ba2e893a0cc2703ffc5a63bf29-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bd7787ba2e893a0cc2703ffc5a63bf29 columnFamilyName cf1 2023-07-24 07:54:52,635 INFO [StoreOpener-bd7787ba2e893a0cc2703ffc5a63bf29-1] regionserver.HStore(310): Store=bd7787ba2e893a0cc2703ffc5a63bf29/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:54:52,636 INFO [StoreOpener-bd7787ba2e893a0cc2703ffc5a63bf29-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:52,637 DEBUG [StoreOpener-bd7787ba2e893a0cc2703ffc5a63bf29-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2 2023-07-24 07:54:52,637 DEBUG [StoreOpener-bd7787ba2e893a0cc2703ffc5a63bf29-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2 2023-07-24 07:54:52,637 INFO [StoreOpener-bd7787ba2e893a0cc2703ffc5a63bf29-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bd7787ba2e893a0cc2703ffc5a63bf29 columnFamilyName cf2 2023-07-24 07:54:52,638 INFO [StoreOpener-bd7787ba2e893a0cc2703ffc5a63bf29-1] regionserver.HStore(310): Store=bd7787ba2e893a0cc2703ffc5a63bf29/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:54:52,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:52,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:52,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:54:52,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:52,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:54:52,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bd7787ba2e893a0cc2703ffc5a63bf29; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63088506, jitterRate=-0.059908002614974976}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:54:52,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bd7787ba2e893a0cc2703ffc5a63bf29: 2023-07-24 07:54:52,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29., pid=104, masterSystemTime=1690185292628 2023-07-24 07:54:52,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:52,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:52,645 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=bd7787ba2e893a0cc2703ffc5a63bf29, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,646 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1690185292645"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185292645"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185292645"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185292645"}]},"ts":"1690185292645"} 2023-07-24 07:54:52,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-07-24 07:54:52,649 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure bd7787ba2e893a0cc2703ffc5a63bf29, server=jenkins-hbase4.apache.org,46543,1690185199091 in 171 msec 2023-07-24 07:54:52,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-07-24 07:54:52,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=bd7787ba2e893a0cc2703ffc5a63bf29, ASSIGN in 327 msec 2023-07-24 07:54:52,651 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 07:54:52,651 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185292651"}]},"ts":"1690185292651"} 2023-07-24 07:54:52,652 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-07-24 07:54:52,654 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 07:54:52,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 374 msec 2023-07-24 07:54:52,788 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:52,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 07:54:52,790 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 07:54:52,790 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:52,790 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 07:54:52,790 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 07:54:52,790 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,791 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,792 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,792 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:52,792 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:52,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:52,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:52,793 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,793 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:52,792 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:52,793 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:52,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:52,794 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:52,794 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:52,794 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:52,795 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:52,795 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:52,795 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:52,795 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:52,795 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:52,795 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:52,795 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:52,795 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:52,795 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. started... 2023-07-24 07:54:52,795 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:52,795 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:52,795 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:52,795 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:52,795 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:52,796 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:52,796 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bd7787ba2e893a0cc2703ffc5a63bf29 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 07:54:52,796 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:52,797 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,798 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,798 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,798 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:52,798 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,798 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:52,807 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/be53a7cd308646c6b0020aa2768a39c6 2023-07-24 07:54:52,822 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/9eb37d058d3d413ba3fad7d9fddb02de 2023-07-24 07:54:52,827 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/be53a7cd308646c6b0020aa2768a39c6 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/be53a7cd308646c6b0020aa2768a39c6 2023-07-24 07:54:52,832 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/be53a7cd308646c6b0020aa2768a39c6, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:54:52,833 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/9eb37d058d3d413ba3fad7d9fddb02de as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/9eb37d058d3d413ba3fad7d9fddb02de 2023-07-24 07:54:52,838 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/9eb37d058d3d413ba3fad7d9fddb02de, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:54:52,839 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for bd7787ba2e893a0cc2703ffc5a63bf29 in 43ms, sequenceid=6, compaction requested=false 2023-07-24 07:54:52,839 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-07-24 07:54:52,840 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bd7787ba2e893a0cc2703ffc5a63bf29: 2023-07-24 07:54:52,841 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:52,841 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:52,841 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:52,841 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,841 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:52,841 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:52,843 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,843 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:52,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:52,843 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,843 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:52,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:52,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:52,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:52,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:52,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:52,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:52,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 07:54:52,846 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5fc896cc[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:52,846 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 07:54:52,846 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,848 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,848 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,848 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,848 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:52,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,848 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:52,849 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:52,849 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:52,848 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:52,849 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:52,849 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:52,849 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:52,849 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:52,851 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,851 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:52,853 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:52,853 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,853 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:52,853 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:52,853 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:52,853 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:52,853 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:52,853 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:52,853 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:52,853 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:52,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:52,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:52,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:52,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:52,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:52,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:52,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:52,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:52,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:52,857 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 07:54:52,857 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 07:54:52,857 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:52,857 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:52,857 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 07:54:52,857 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:52,858 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,859 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,859 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,859 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:52,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,859 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:52,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,859 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,859 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:52,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:52,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:52,859 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:52,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,860 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:52,860 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:52,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:52,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:52,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:52,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:52,862 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,862 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:52,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:52,863 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:52,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:52,868 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:52,868 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:52,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:52,869 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:52,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:52,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:52,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:52,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 07:54:52,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:52,869 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:52,868 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:52,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:52,868 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:52,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:52,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:52,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:52,869 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:52,869 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:52,869 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:52,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:52,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:52,869 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:52,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:52,869 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:52,870 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:52,870 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,870 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,870 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:52,870 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,870 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:52,870 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,870 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,870 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,970 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:52,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:52,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:52,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-24 07:54:52,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 07:54:52,986 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 07:54:52,986 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:52,986 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 07:54:52,986 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 07:54:52,986 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,986 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,988 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,988 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:52,988 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:52,988 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:52,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:52,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:52,988 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:52,988 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:52,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:52,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:52,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:52,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:52,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:52,989 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:52,990 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:52,990 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:52,990 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:52,990 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:52,990 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:52,990 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:52,990 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:52,990 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:52,990 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:52,990 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:52,990 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:52,990 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:52,990 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:52,990 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:52,991 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:52,991 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:52,991 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:52,991 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:52,991 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:52,991 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,992 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:52,992 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:52,992 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:52,992 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. started... 2023-07-24 07:54:52,992 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bd7787ba2e893a0cc2703ffc5a63bf29 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:54:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:52,997 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,003 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/e3a5f19e4ead4f32b79b7877661cf3e2 2023-07-24 07:54:53,015 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/179aa92a09c1413689b72f214331fc0d 2023-07-24 07:54:53,020 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/e3a5f19e4ead4f32b79b7877661cf3e2 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/e3a5f19e4ead4f32b79b7877661cf3e2 2023-07-24 07:54:53,024 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/e3a5f19e4ead4f32b79b7877661cf3e2, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:54:53,025 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/179aa92a09c1413689b72f214331fc0d as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/179aa92a09c1413689b72f214331fc0d 2023-07-24 07:54:53,029 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/179aa92a09c1413689b72f214331fc0d, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:54:53,030 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bd7787ba2e893a0cc2703ffc5a63bf29 in 38ms, sequenceid=19, compaction requested=false 2023-07-24 07:54:53,030 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bd7787ba2e893a0cc2703ffc5a63bf29: 2023-07-24 07:54:53,031 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:53,031 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:53,031 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:53,031 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,031 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,031 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,032 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,032 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,033 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,033 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 07:54:53,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28ae5943[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:53,035 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 07:54:53,035 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,036 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,036 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,036 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,036 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,036 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:53,036 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,036 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,036 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,036 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:53,036 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,036 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,037 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:53,037 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,037 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:53,038 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,038 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,038 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,038 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,040 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,040 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,040 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,040 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,040 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,040 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:53,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 07:54:53,043 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 07:54:53,043 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:53,043 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:53,043 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 07:54:53,043 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:53,045 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,045 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,045 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,045 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,045 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,045 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,045 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,046 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,046 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,047 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,048 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,048 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,049 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,052 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,052 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,052 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,052 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,052 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,052 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,052 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,052 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,052 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,052 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 07:54:53,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:53,052 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,053 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:53,053 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,053 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:53,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,053 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,053 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,053 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,053 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,053 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,053 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,153 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:53,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:53,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:53,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-24 07:54:53,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 07:54:53,168 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 07:54:53,168 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,169 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 07:54:53,169 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 07:54:53,169 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,169 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,170 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,170 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,171 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,171 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:53,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,171 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,172 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,172 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,172 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,172 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,172 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,172 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,173 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,172 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:53,173 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:53,173 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:53,173 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. started... 2023-07-24 07:54:53,173 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bd7787ba2e893a0cc2703ffc5a63bf29 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:54:53,176 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,176 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,176 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,176 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,176 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,176 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,185 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/b90009aca2ea44cd89dcef5c160ee52b 2023-07-24 07:54:53,205 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/e2dca9185df642dfb9a9a7f4a3f1c46d 2023-07-24 07:54:53,209 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/b90009aca2ea44cd89dcef5c160ee52b as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/b90009aca2ea44cd89dcef5c160ee52b 2023-07-24 07:54:53,213 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/b90009aca2ea44cd89dcef5c160ee52b, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:54:53,214 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/e2dca9185df642dfb9a9a7f4a3f1c46d as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/e2dca9185df642dfb9a9a7f4a3f1c46d 2023-07-24 07:54:53,217 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/e2dca9185df642dfb9a9a7f4a3f1c46d, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:54:53,218 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bd7787ba2e893a0cc2703ffc5a63bf29 in 45ms, sequenceid=32, compaction requested=true 2023-07-24 07:54:53,218 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bd7787ba2e893a0cc2703ffc5a63bf29: 2023-07-24 07:54:53,218 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:53,219 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:53,219 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:53,219 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,219 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,219 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,220 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,220 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,220 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,221 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 07:54:53,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37165b24[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:53,222 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 07:54:53,222 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,223 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,223 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,223 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,224 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,224 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,224 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:53,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,224 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,224 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:53,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,224 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,224 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,224 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:53,224 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,224 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,224 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:53,225 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,225 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,225 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,228 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,228 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:53,231 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 07:54:53,231 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 07:54:53,231 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:53,231 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:53,231 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 07:54:53,231 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:53,233 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,233 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,233 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,233 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,233 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,233 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,233 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,234 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,234 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,235 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,236 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,237 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,237 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,243 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,243 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,243 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,243 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,243 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,243 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 07:54:53,243 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:53,243 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,243 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,243 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:53,244 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:53,243 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,243 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,244 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,244 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,244 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,244 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,244 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,244 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,244 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,244 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,244 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,344 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:53,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:53,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:53,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-24 07:54:53,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 07:54:53,360 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 07:54:53,361 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,361 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 07:54:53,361 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 07:54:53,361 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,361 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,363 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,364 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,363 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,363 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,364 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,364 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:53,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,364 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,364 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,365 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,365 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,365 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,365 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,365 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,366 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:53,366 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:53,366 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. started... 2023-07-24 07:54:53,366 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bd7787ba2e893a0cc2703ffc5a63bf29 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:54:53,367 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,367 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,368 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,368 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,368 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,368 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,377 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/33d2880d6f8643a38edec93c43276e1a 2023-07-24 07:54:53,393 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/276ad84666ce4a03b301c91b0a84c656 2023-07-24 07:54:53,397 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/33d2880d6f8643a38edec93c43276e1a as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/33d2880d6f8643a38edec93c43276e1a 2023-07-24 07:54:53,401 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/33d2880d6f8643a38edec93c43276e1a, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:54:53,402 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/276ad84666ce4a03b301c91b0a84c656 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/276ad84666ce4a03b301c91b0a84c656 2023-07-24 07:54:53,405 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/276ad84666ce4a03b301c91b0a84c656, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:54:53,406 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bd7787ba2e893a0cc2703ffc5a63bf29 in 40ms, sequenceid=45, compaction requested=true 2023-07-24 07:54:53,406 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bd7787ba2e893a0cc2703ffc5a63bf29: 2023-07-24 07:54:53,406 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:53,406 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:53,406 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:53,407 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,407 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,407 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,408 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,408 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,409 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,409 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 07:54:53,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47bb9dc7[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:53,411 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 07:54:53,411 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,412 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,412 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,412 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,412 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,412 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,412 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,413 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:53,412 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,412 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:53,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,413 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:53,415 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,415 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,415 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,415 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,415 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,415 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,416 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,416 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,416 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,416 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:53,420 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 07:54:53,420 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 07:54:53,420 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:53,420 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:53,420 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 07:54:53,420 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:53,421 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,421 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,422 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,422 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,422 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,422 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,422 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,422 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,423 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,423 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,425 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,425 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,426 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,428 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,429 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,429 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,429 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,429 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,429 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,429 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 07:54:53,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:53,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,429 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,429 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,429 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,429 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:53,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,429 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,430 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:53,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,430 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,430 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,430 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,430 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,430 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,430 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,430 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,530 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:53,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:53,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:53,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-24 07:54:53,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 07:54:53,544 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 07:54:53,544 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,545 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 07:54:53,545 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 07:54:53,545 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,545 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,547 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,547 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,547 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,547 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,547 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,548 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:53,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,549 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,549 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,549 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,550 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,549 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,550 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,550 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,550 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,550 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,550 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,550 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,550 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:53,550 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:53,550 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,550 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. started... 2023-07-24 07:54:53,550 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:53,550 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,550 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bd7787ba2e893a0cc2703ffc5a63bf29 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:54:53,550 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,551 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,552 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,552 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,553 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,553 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,553 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,561 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/9d392fc233f24d81b944153ba663eb9b 2023-07-24 07:54:53,572 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/7892b8bea00641cbacfa3ad69c5ecff0 2023-07-24 07:54:53,576 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/9d392fc233f24d81b944153ba663eb9b as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/9d392fc233f24d81b944153ba663eb9b 2023-07-24 07:54:53,581 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/9d392fc233f24d81b944153ba663eb9b, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:54:53,582 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/7892b8bea00641cbacfa3ad69c5ecff0 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/7892b8bea00641cbacfa3ad69c5ecff0 2023-07-24 07:54:53,586 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/7892b8bea00641cbacfa3ad69c5ecff0, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:54:53,587 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bd7787ba2e893a0cc2703ffc5a63bf29 in 37ms, sequenceid=58, compaction requested=true 2023-07-24 07:54:53,587 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bd7787ba2e893a0cc2703ffc5a63bf29: 2023-07-24 07:54:53,587 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:53,587 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:53,588 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:53,588 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,588 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,588 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,589 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,589 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,589 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,589 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,591 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 07:54:53,591 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5cde1d0a[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:53,591 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 07:54:53,591 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,593 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,593 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,593 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:53,594 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,594 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:53,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:53,594 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:53,603 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,603 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,603 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,607 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,607 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,607 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,607 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,607 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,607 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,607 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:53,610 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 07:54:53,610 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 07:54:53,610 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:53,610 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:53,610 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 07:54:53,611 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:53,613 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,613 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,613 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,613 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,613 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,614 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,614 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,614 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,614 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,615 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,617 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,617 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,617 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,620 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,620 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,620 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,620 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,620 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,620 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,620 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,621 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 07:54:53,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:53,620 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,620 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,621 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,621 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,621 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:53,621 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,621 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:53,621 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,622 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,622 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,622 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,622 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,722 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:53,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:53,734 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:53,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-24 07:54:53,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 07:54:53,735 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 07:54:53,736 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,736 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 07:54:53,736 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 07:54:53,736 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,736 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,737 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,737 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,737 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,737 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,738 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,738 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:53,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,738 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,739 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:53,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,739 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,739 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 07:54:53,739 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,739 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,739 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,739 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,739 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:53,739 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,739 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. started... 2023-07-24 07:54:53,739 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,740 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing bd7787ba2e893a0cc2703ffc5a63bf29 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:54:53,741 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,743 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,743 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,743 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,743 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,743 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,748 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/7496c7e0a91f4899b25b9f8dbee5d6a9 2023-07-24 07:54:53,760 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/ef62f27a3f774cd88526b71f5662655a 2023-07-24 07:54:53,763 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/7496c7e0a91f4899b25b9f8dbee5d6a9 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/7496c7e0a91f4899b25b9f8dbee5d6a9 2023-07-24 07:54:53,767 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/7496c7e0a91f4899b25b9f8dbee5d6a9, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:54:53,768 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/ef62f27a3f774cd88526b71f5662655a as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/ef62f27a3f774cd88526b71f5662655a 2023-07-24 07:54:53,771 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/ef62f27a3f774cd88526b71f5662655a, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:54:53,772 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for bd7787ba2e893a0cc2703ffc5a63bf29 in 32ms, sequenceid=71, compaction requested=true 2023-07-24 07:54:53,772 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bd7787ba2e893a0cc2703ffc5a63bf29: 2023-07-24 07:54:53,772 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:53,772 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:53,772 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:53,772 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,773 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 07:54:53,773 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 07:54:53,774 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,774 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,774 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,774 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:53,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 07:54:53,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6e03393[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:53,776 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 07:54:53,776 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,777 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,777 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,777 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,778 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,778 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:53,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,778 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,778 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,778 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,778 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,778 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:53,778 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 07:54:53,778 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 07:54:53,778 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:53,778 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,780 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,781 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,781 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 07:54:53,781 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,781 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,781 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,781 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,782 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 07:54:53,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:53,785 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 07:54:53,785 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 07:54:53,785 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:53,785 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:53,785 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 07:54:53,785 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:53,787 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,787 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,787 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,787 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,787 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,787 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,787 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,787 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:53,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:53,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,788 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,788 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:53,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,789 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,790 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:53,790 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 07:54:53,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,793 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,793 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,793 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,793 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:53,793 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,793 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:53,793 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,793 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 07:54:53,793 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:53,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:53,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:53,793 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,793 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:53,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,793 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:53,793 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:53,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,793 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:53,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:53,794 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,794 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 07:54:53,794 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:53,794 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,794 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:53,794 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,794 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 07:54:53,794 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 07:54:53,894 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:53,895 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:53,901 INFO [Listener at localhost/42537] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,902 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.StoreUtils(129): cannot split hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/e3a5f19e4ead4f32b79b7877661cf3e2 because midkey is the same as first or last row 2023-07-24 07:54:53,903 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.StoreUtils(129): cannot split hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/179aa92a09c1413689b72f214331fc0d because midkey is the same as first or last row 2023-07-24 07:54:53,903 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.HRegion(2745): Flushing bd7787ba2e893a0cc2703ffc5a63bf29 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 07:54:53,911 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/d07c4e4e3373401989ff28cee91da712 2023-07-24 07:54:53,923 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/c7646232dc8b4fbd9f9b4283e279992f 2023-07-24 07:54:53,927 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf1/d07c4e4e3373401989ff28cee91da712 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/d07c4e4e3373401989ff28cee91da712 2023-07-24 07:54:53,930 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/d07c4e4e3373401989ff28cee91da712, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:54:53,931 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/.tmp/cf2/c7646232dc8b4fbd9f9b4283e279992f as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/c7646232dc8b4fbd9f9b4283e279992f 2023-07-24 07:54:53,935 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/c7646232dc8b4fbd9f9b4283e279992f, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:54:53,935 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for bd7787ba2e893a0cc2703ffc5a63bf29 in 32ms, sequenceid=82, compaction requested=true 2023-07-24 07:54:53,936 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.HRegion(2446): Flush status journal for bd7787ba2e893a0cc2703ffc5a63bf29: 2023-07-24 07:54:53,936 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.StoreUtils(129): cannot split hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/e3a5f19e4ead4f32b79b7877661cf3e2 because midkey is the same as first or last row 2023-07-24 07:54:53,937 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46543] regionserver.StoreUtils(129): cannot split hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/179aa92a09c1413689b72f214331fc0d because midkey is the same as first or last row 2023-07-24 07:54:53,937 DEBUG [Listener at localhost/42537] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,937 DEBUG [Listener at localhost/42537] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-07-24 07:54:53,937 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:54:53,939 INFO [Listener at localhost/42537] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-07-24 07:54:53,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-07-24 07:54:53,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:53,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-24 07:54:53,942 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185293942"}]},"ts":"1690185293942"} 2023-07-24 07:54:53,943 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-07-24 07:54:53,945 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-07-24 07:54:53,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=bd7787ba2e893a0cc2703ffc5a63bf29, UNASSIGN}] 2023-07-24 07:54:53,947 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=bd7787ba2e893a0cc2703ffc5a63bf29, UNASSIGN 2023-07-24 07:54:53,947 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=bd7787ba2e893a0cc2703ffc5a63bf29, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:53,947 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1690185293947"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185293947"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185293947"}]},"ts":"1690185293947"} 2023-07-24 07:54:53,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure bd7787ba2e893a0cc2703ffc5a63bf29, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:54:54,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-24 07:54:54,101 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:54,101 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bd7787ba2e893a0cc2703ffc5a63bf29, disabling compactions & flushes 2023-07-24 07:54:54,101 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:54,101 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:54,101 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. after waiting 0 ms 2023-07-24 07:54:54,101 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:54,105 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 07:54:54,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:54:54,106 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29. 2023-07-24 07:54:54,106 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bd7787ba2e893a0cc2703ffc5a63bf29: 2023-07-24 07:54:54,107 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:54,107 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=bd7787ba2e893a0cc2703ffc5a63bf29, regionState=CLOSED 2023-07-24 07:54:54,108 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1690185294107"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185294107"}]},"ts":"1690185294107"} 2023-07-24 07:54:54,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-07-24 07:54:54,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure bd7787ba2e893a0cc2703ffc5a63bf29, server=jenkins-hbase4.apache.org,46543,1690185199091 in 161 msec 2023-07-24 07:54:54,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-07-24 07:54:54,111 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=bd7787ba2e893a0cc2703ffc5a63bf29, UNASSIGN in 164 msec 2023-07-24 07:54:54,112 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185294112"}]},"ts":"1690185294112"} 2023-07-24 07:54:54,113 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-07-24 07:54:54,114 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-07-24 07:54:54,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 176 msec 2023-07-24 07:54:54,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-24 07:54:54,245 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-07-24 07:54:54,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-07-24 07:54:54,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:54,248 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:54,248 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:54,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-07-24 07:54:54,251 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:54,253 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/recovered.edits] 2023-07-24 07:54:54,256 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/33d2880d6f8643a38edec93c43276e1a to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/33d2880d6f8643a38edec93c43276e1a 2023-07-24 07:54:54,257 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/7496c7e0a91f4899b25b9f8dbee5d6a9 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/7496c7e0a91f4899b25b9f8dbee5d6a9 2023-07-24 07:54:54,258 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/9d392fc233f24d81b944153ba663eb9b to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/9d392fc233f24d81b944153ba663eb9b 2023-07-24 07:54:54,259 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/b90009aca2ea44cd89dcef5c160ee52b to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/b90009aca2ea44cd89dcef5c160ee52b 2023-07-24 07:54:54,261 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/be53a7cd308646c6b0020aa2768a39c6 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/be53a7cd308646c6b0020aa2768a39c6 2023-07-24 07:54:54,262 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/d07c4e4e3373401989ff28cee91da712 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/d07c4e4e3373401989ff28cee91da712 2023-07-24 07:54:54,263 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/e3a5f19e4ead4f32b79b7877661cf3e2 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf1/e3a5f19e4ead4f32b79b7877661cf3e2 2023-07-24 07:54:54,266 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/179aa92a09c1413689b72f214331fc0d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/179aa92a09c1413689b72f214331fc0d 2023-07-24 07:54:54,267 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/276ad84666ce4a03b301c91b0a84c656 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/276ad84666ce4a03b301c91b0a84c656 2023-07-24 07:54:54,268 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/7892b8bea00641cbacfa3ad69c5ecff0 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/7892b8bea00641cbacfa3ad69c5ecff0 2023-07-24 07:54:54,269 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/9eb37d058d3d413ba3fad7d9fddb02de to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/9eb37d058d3d413ba3fad7d9fddb02de 2023-07-24 07:54:54,271 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/c7646232dc8b4fbd9f9b4283e279992f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/c7646232dc8b4fbd9f9b4283e279992f 2023-07-24 07:54:54,272 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/e2dca9185df642dfb9a9a7f4a3f1c46d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/e2dca9185df642dfb9a9a7f4a3f1c46d 2023-07-24 07:54:54,273 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/ef62f27a3f774cd88526b71f5662655a to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/cf2/ef62f27a3f774cd88526b71f5662655a 2023-07-24 07:54:54,275 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/recovered.edits/85.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29/recovered.edits/85.seqid 2023-07-24 07:54:54,276 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testInvalidSplitKey/bd7787ba2e893a0cc2703ffc5a63bf29 2023-07-24 07:54:54,276 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-07-24 07:54:54,278 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:54,279 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-07-24 07:54:54,283 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-07-24 07:54:54,283 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:54,283 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-07-24 07:54:54,284 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185294283"}]},"ts":"9223372036854775807"} 2023-07-24 07:54:54,285 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-24 07:54:54,285 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => bd7787ba2e893a0cc2703ffc5a63bf29, NAME => 'testInvalidSplitKey,,1690185292280.bd7787ba2e893a0cc2703ffc5a63bf29.', STARTKEY => '', ENDKEY => ''}] 2023-07-24 07:54:54,285 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-07-24 07:54:54,285 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690185294285"}]},"ts":"9223372036854775807"} 2023-07-24 07:54:54,287 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-07-24 07:54:54,289 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:54,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 44 msec 2023-07-24 07:54:54,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-07-24 07:54:54,350 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-07-24 07:54:54,364 INFO [Listener at localhost/42537] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=371 (was 351) - Thread LEAK? -, OpenFileDescriptor=748 (was 659) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=162 (was 168), ProcessCount=171 (was 171), AvailableMemoryMB=8888 (was 8902) 2023-07-24 07:54:54,378 INFO [Listener at localhost/42537] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=371, OpenFileDescriptor=748, MaxFileDescriptor=60000, SystemLoadAverage=162, ProcessCount=171, AvailableMemoryMB=8887 2023-07-24 07:54:54,378 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:54:54,380 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 07:54:54,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 07:54:54,382 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:54,383 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 07:54:54,384 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 07:54:54,386 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:54,387 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d empty. 2023-07-24 07:54:54,388 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:54,388 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-07-24 07:54:54,398 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-07-24 07:54:54,399 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 63edc6e611d2d72dc562948ea214a40d, NAME => 'testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp 2023-07-24 07:54:54,413 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:54:54,413 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 63edc6e611d2d72dc562948ea214a40d, disabling compactions & flushes 2023-07-24 07:54:54,413 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:54,413 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:54,413 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. after waiting 0 ms 2023-07-24 07:54:54,413 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:54,413 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:54,413 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 63edc6e611d2d72dc562948ea214a40d: 2023-07-24 07:54:54,416 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 07:54:54,417 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185294416"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185294416"}]},"ts":"1690185294416"} 2023-07-24 07:54:54,418 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 07:54:54,418 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 07:54:54,418 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185294418"}]},"ts":"1690185294418"} 2023-07-24 07:54:54,419 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-07-24 07:54:54,424 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:54:54,425 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:54:54,425 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:54:54,425 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:54:54,425 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:54:54,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, ASSIGN}] 2023-07-24 07:54:54,426 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, ASSIGN 2023-07-24 07:54:54,427 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:54:54,577 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:54:54,578 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=63edc6e611d2d72dc562948ea214a40d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,579 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185294578"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185294578"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185294578"}]},"ts":"1690185294578"} 2023-07-24 07:54:54,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:54:54,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:54,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 63edc6e611d2d72dc562948ea214a40d, NAME => 'testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:54:54,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:54:54,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:54,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:54:54,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:54,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:54,736 INFO [StoreOpener-63edc6e611d2d72dc562948ea214a40d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:54,737 DEBUG [StoreOpener-63edc6e611d2d72dc562948ea214a40d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1 2023-07-24 07:54:54,738 DEBUG [StoreOpener-63edc6e611d2d72dc562948ea214a40d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1 2023-07-24 07:54:54,738 INFO [StoreOpener-63edc6e611d2d72dc562948ea214a40d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 63edc6e611d2d72dc562948ea214a40d columnFamilyName cf1 2023-07-24 07:54:54,738 INFO [StoreOpener-63edc6e611d2d72dc562948ea214a40d-1] regionserver.HStore(310): Store=63edc6e611d2d72dc562948ea214a40d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:54:54,738 INFO [StoreOpener-63edc6e611d2d72dc562948ea214a40d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:54,739 DEBUG [StoreOpener-63edc6e611d2d72dc562948ea214a40d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2 2023-07-24 07:54:54,740 DEBUG [StoreOpener-63edc6e611d2d72dc562948ea214a40d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2 2023-07-24 07:54:54,740 INFO [StoreOpener-63edc6e611d2d72dc562948ea214a40d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 63edc6e611d2d72dc562948ea214a40d columnFamilyName cf2 2023-07-24 07:54:54,740 INFO [StoreOpener-63edc6e611d2d72dc562948ea214a40d-1] regionserver.HStore(310): Store=63edc6e611d2d72dc562948ea214a40d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:54:54,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:54,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:54,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:54:54,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:54,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:54:54,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 63edc6e611d2d72dc562948ea214a40d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60684325, jitterRate=-0.09573309123516083}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:54:54,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 63edc6e611d2d72dc562948ea214a40d: 2023-07-24 07:54:54,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d., pid=111, masterSystemTime=1690185294732 2023-07-24 07:54:54,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:54,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:54,749 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=63edc6e611d2d72dc562948ea214a40d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,749 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185294749"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185294749"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185294749"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185294749"}]},"ts":"1690185294749"} 2023-07-24 07:54:54,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-07-24 07:54:54,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 in 170 msec 2023-07-24 07:54:54,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-07-24 07:54:54,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, ASSIGN in 327 msec 2023-07-24 07:54:54,754 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 07:54:54,754 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185294754"}]},"ts":"1690185294754"} 2023-07-24 07:54:54,755 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-07-24 07:54:54,757 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 07:54:54,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 376 msec 2023-07-24 07:54:54,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:54,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 07:54:54,891 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:54,891 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:54,892 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 07:54:54,892 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:54:54,892 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,892 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,893 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,893 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:54,893 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:54,893 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:54,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:54,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:54,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:54,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:54,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:54,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:54,894 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,894 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:54,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:54,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,895 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:54,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:54,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:54,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,895 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:54,895 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:54,895 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:54,895 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:54,895 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:54,895 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:54,896 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:54,896 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:54,896 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:54,896 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. started... 2023-07-24 07:54:54,897 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 63edc6e611d2d72dc562948ea214a40d 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 07:54:54,897 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,898 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,898 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,898 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:54,898 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,898 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:54,908 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/99612eba074c4839ac421fe8b8d405d5 2023-07-24 07:54:54,919 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/d12defefae7341b5910a940e956e7f48 2023-07-24 07:54:54,923 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/99612eba074c4839ac421fe8b8d405d5 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5 2023-07-24 07:54:54,927 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:54:54,927 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/d12defefae7341b5910a940e956e7f48 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48 2023-07-24 07:54:54,931 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:54:54,932 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 63edc6e611d2d72dc562948ea214a40d in 35ms, sequenceid=6, compaction requested=false 2023-07-24 07:54:54,932 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:54,932 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 63edc6e611d2d72dc562948ea214a40d: 2023-07-24 07:54:54,932 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:54,932 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:54,932 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:54,932 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,932 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:54,932 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:54,934 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,934 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:54,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:54,934 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,934 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:54,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:54,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:54,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:54,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:54,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:54,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:54,936 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 07:54:54,936 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32a8710[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:54,936 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:54:54,936 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,938 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,938 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,938 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,938 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:54,938 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:54,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,938 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:54,938 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:54,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,938 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,938 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:54,938 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:54,938 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:54,938 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:54,938 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:54,938 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:54,941 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:54,941 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:54,941 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:54,941 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,941 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:54,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,941 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:54,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:54,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:54,941 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:54,942 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:54,942 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:54,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:54,942 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:54,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:54,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:54,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:54,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:54,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:54,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:54,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:54,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:54,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:54,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:54:54,945 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 07:54:54,945 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:54,945 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:54,945 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 07:54:54,945 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:54,947 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,947 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,947 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,947 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:54,947 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:54,947 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,947 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:54,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:54,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,947 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:54,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,948 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:54,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:54,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:54,948 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:54,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:54,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:54,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,950 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:54,950 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:54,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:54,951 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:54,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:54,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:54,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:54,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:54,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:54,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:54,954 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,954 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution because node does not exist (not an error) 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:54,954 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:54,954 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:54,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:54,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:54,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:54,954 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:54,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:54,955 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:54,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:54,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:54,955 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:54,955 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:54,955 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:54,955 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:54,955 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:54,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:54,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:54,955 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,055 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:55,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:55,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:55,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 07:54:55,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 07:54:55,069 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:55,069 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,069 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 07:54:55,069 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:54:55,069 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,069 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,071 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,071 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,071 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,071 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,071 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,071 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:55,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,071 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,072 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,072 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,072 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,072 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,073 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,073 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,073 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,073 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,073 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,073 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,073 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,073 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,073 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,073 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,073 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:55,073 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:55,073 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:55,073 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. started... 2023-07-24 07:54:55,073 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 63edc6e611d2d72dc562948ea214a40d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:54:55,074 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,075 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,075 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,077 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,078 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,078 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,078 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,078 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,078 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,078 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,078 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,078 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,080 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,080 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,080 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,083 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/19b141de763a465397e1aa9b2273b8f3 2023-07-24 07:54:55,120 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/a32fa3ab81384813881ecde578c4e4d0 2023-07-24 07:54:55,124 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/19b141de763a465397e1aa9b2273b8f3 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3 2023-07-24 07:54:55,128 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:54:55,128 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/a32fa3ab81384813881ecde578c4e4d0 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0 2023-07-24 07:54:55,132 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:54:55,132 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 63edc6e611d2d72dc562948ea214a40d in 59ms, sequenceid=19, compaction requested=false 2023-07-24 07:54:55,132 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 63edc6e611d2d72dc562948ea214a40d: 2023-07-24 07:54:55,132 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:55,133 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:55,133 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,133 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,133 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,137 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,137 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,138 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,138 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 07:54:55,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18e47717[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:55,140 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:54:55,140 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,141 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,141 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,141 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,141 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,141 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,141 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:55,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,141 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,141 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,141 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,142 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:55,142 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,142 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,142 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:55,142 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:55,143 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,143 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,143 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,144 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,144 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,144 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,144 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,144 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,144 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,144 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:55,147 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:54:55,147 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 07:54:55,147 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:55,147 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:55,147 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,147 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:55,149 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,149 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,150 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,150 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,150 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,150 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,150 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,150 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,151 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,151 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,153 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,153 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,153 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,156 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,156 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,156 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,156 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,156 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:55,156 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:55,156 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,156 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,156 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,156 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,157 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:55,156 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,157 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,157 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:55,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,157 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,157 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,157 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,157 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,157 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,157 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,157 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,257 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:55,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:55,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:55,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 07:54:55,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 07:54:55,271 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:55,271 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,271 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 07:54:55,271 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:54:55,271 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,271 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,272 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,272 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,272 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,272 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,273 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,273 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,274 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,274 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,274 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,274 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,274 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,275 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,275 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,275 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:55,275 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:55,275 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:55,275 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. started... 2023-07-24 07:54:55,275 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 63edc6e611d2d72dc562948ea214a40d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:54:55,276 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,276 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,277 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,277 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,277 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,277 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,284 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/3d44f2d8f3794b469861a10f9b1150d2 2023-07-24 07:54:55,294 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/ca7683188a754a688ee85e220c336277 2023-07-24 07:54:55,298 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/3d44f2d8f3794b469861a10f9b1150d2 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2 2023-07-24 07:54:55,301 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:54:55,302 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/ca7683188a754a688ee85e220c336277 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277 2023-07-24 07:54:55,305 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:54:55,306 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 63edc6e611d2d72dc562948ea214a40d in 31ms, sequenceid=32, compaction requested=true 2023-07-24 07:54:55,306 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 63edc6e611d2d72dc562948ea214a40d: 2023-07-24 07:54:55,306 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:55,306 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:55,306 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:55,306 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,306 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,306 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,309 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,309 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,309 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,309 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 07:54:55,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@347e1f4c[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:55,311 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:54:55,311 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,312 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,312 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,312 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,313 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,313 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:55,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:55,314 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,314 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,314 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,315 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,315 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,315 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,315 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,315 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,315 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,315 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:55,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:54:55,318 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 07:54:55,318 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:55,318 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:55,318 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,318 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:55,321 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,321 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,321 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,321 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,321 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,321 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,321 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,321 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,322 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,322 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,324 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,324 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,324 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,329 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,329 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,329 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,329 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,329 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,329 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,329 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,329 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,329 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,330 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:55,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:55,329 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,330 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,330 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,330 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,330 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,330 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,330 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,330 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:55,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,330 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:55,330 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,431 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:55,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:55,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:55,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 07:54:55,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 07:54:55,444 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:55,444 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,444 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 07:54:55,444 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:54:55,445 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,445 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,446 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,446 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,446 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,446 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,447 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,447 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:55,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,448 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,448 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,448 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:55,449 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:55,449 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:55,449 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. started... 2023-07-24 07:54:55,449 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 63edc6e611d2d72dc562948ea214a40d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:54:55,451 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,452 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,452 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,452 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,452 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,452 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,458 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/d7d33c4ba08947f8b752c158aa4b9c59 2023-07-24 07:54:55,469 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/93c6c7a1e37844e18310d5d23ed7fbfb 2023-07-24 07:54:55,473 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/d7d33c4ba08947f8b752c158aa4b9c59 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59 2023-07-24 07:54:55,477 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:54:55,478 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/93c6c7a1e37844e18310d5d23ed7fbfb as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb 2023-07-24 07:54:55,482 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:54:55,483 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 63edc6e611d2d72dc562948ea214a40d in 33ms, sequenceid=45, compaction requested=true 2023-07-24 07:54:55,483 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 63edc6e611d2d72dc562948ea214a40d: 2023-07-24 07:54:55,483 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:55,483 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:55,483 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:55,483 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,483 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,483 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,484 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,484 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,485 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,485 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 07:54:55,486 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3305cefb[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:55,486 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:54:55,487 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,488 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,488 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,488 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,488 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,488 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,488 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:55,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,488 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,488 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,488 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,488 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,488 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:55,488 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:55,488 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,488 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:55,490 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,490 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,490 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,490 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,490 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,490 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,490 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,490 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,490 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,490 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:55,493 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:54:55,493 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 07:54:55,494 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:55,494 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:55,494 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,494 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:55,495 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,495 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,495 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,495 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,495 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,495 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,495 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,496 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,496 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,496 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,498 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,498 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,498 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,501 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,501 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,501 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:55,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:55,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,502 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:55,502 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:55,506 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,506 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,506 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,506 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,506 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,506 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,507 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,507 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,507 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,507 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,507 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,507 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,507 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,507 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,507 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,602 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:55,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:55,611 ERROR [BP-80517894-172.31.14.131-1690185194253 heartbeating to localhost/127.0.0.1:34211] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-80517894-172.31.14.131-1690185194253 (Datanode Uuid cda8ebfd-f055-417e-8edc-61fd19134adf) service to localhost/127.0.0.1:34211 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-24 07:54:55,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:55,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 07:54:55,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 07:54:55,618 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:55,618 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,618 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 07:54:55,618 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:54:55,618 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,618 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,620 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,620 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,620 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,620 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,620 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,620 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:55,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,621 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,621 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,621 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,622 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,622 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,622 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,623 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,623 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:55,623 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,623 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. started... 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,623 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 63edc6e611d2d72dc562948ea214a40d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,623 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,626 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,626 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,626 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,627 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,627 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,627 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,637 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/29e5c4e4f727419b83f9f455dbf87c84 2023-07-24 07:54:55,655 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/30b4d485121341688f91345ed26d6627 2023-07-24 07:54:55,660 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/29e5c4e4f727419b83f9f455dbf87c84 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84 2023-07-24 07:54:55,664 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:54:55,664 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/30b4d485121341688f91345ed26d6627 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627 2023-07-24 07:54:55,668 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:54:55,669 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 63edc6e611d2d72dc562948ea214a40d in 46ms, sequenceid=58, compaction requested=true 2023-07-24 07:54:55,669 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 63edc6e611d2d72dc562948ea214a40d: 2023-07-24 07:54:55,669 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:55,670 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:55,670 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:55,670 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,670 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,670 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,679 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,679 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,680 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,680 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 07:54:55,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6440dd8c[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:55,682 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:54:55,682 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,684 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,684 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,684 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,684 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,685 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,685 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,685 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,685 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:55,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,685 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:55,685 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,685 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,685 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,685 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:55,685 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:55,686 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,686 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,686 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,687 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,687 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,687 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,687 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,687 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,687 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,687 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:55,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:54:55,690 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 07:54:55,690 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:55,690 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:55,690 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,690 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:55,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,693 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,693 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,694 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,695 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,696 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,696 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,699 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,699 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,699 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,699 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,699 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,699 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:55,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:55,699 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,699 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,699 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,699 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,700 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:55,700 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,700 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:55,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,700 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,700 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,700 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,700 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,700 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,700 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,700 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,800 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:55,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:55,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:54:55,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 07:54:55,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 07:54:55,813 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:55,813 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,813 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 07:54:55,813 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 07:54:55,814 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,814 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,815 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,815 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,815 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,815 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,815 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,816 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:54:55,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,816 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,816 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,816 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:54:55,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,817 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,817 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 07:54:55,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,817 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,817 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:55,817 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. started... 2023-07-24 07:54:55,818 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 63edc6e611d2d72dc562948ea214a40d 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:54:55,819 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,819 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,819 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,819 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,820 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,820 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,826 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/02647ff08c7b4d408c45334940799d3e 2023-07-24 07:54:55,837 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/d531a392fa1e475bac471f0201838a5f 2023-07-24 07:54:55,842 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/02647ff08c7b4d408c45334940799d3e as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e 2023-07-24 07:54:55,846 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:54:55,847 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/d531a392fa1e475bac471f0201838a5f as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f 2023-07-24 07:54:55,851 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:54:55,852 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 63edc6e611d2d72dc562948ea214a40d in 34ms, sequenceid=71, compaction requested=true 2023-07-24 07:54:55,852 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 63edc6e611d2d72dc562948ea214a40d: 2023-07-24 07:54:55,852 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:55,852 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:54:55,852 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:54:55,852 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,852 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 07:54:55,852 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 07:54:55,855 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,855 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,855 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,855 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:54:55,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,857 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 07:54:55,857 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5fa742dc[Count = 0] remaining members to acquire global barrier 2023-07-24 07:54:55,857 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 07:54:55,857 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,859 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,859 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,859 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:54:55,859 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,859 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 07:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 07:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:54:55,859 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,861 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 07:54:55,862 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,862 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 07:54:55,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:54:55,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 07:54:55,866 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 07:54:55,866 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:54:55,866 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:54:55,866 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 07:54:55,866 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:54:55,868 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,868 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,868 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,869 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,868 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,868 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,869 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,869 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:54:55,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,870 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:54:55,870 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:54:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,872 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,872 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:54:55,872 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 07:54:55,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,877 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,877 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,877 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,877 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,877 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:54:55,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,878 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,877 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,877 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:54:55,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:54:55,878 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:54:55,878 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,878 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,878 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:54:55,878 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,878 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:54:55,878 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:54:55,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 07:54:55,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,878 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:54:55,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:54:55,878 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:55,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,879 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:54:55,879 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 07:54:55,879 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:54:55,879 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 07:54:55,979 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:54:55,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:54:56,238 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-24 07:54:56,238 DEBUG [Listener at localhost/42537] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:56,239 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,240 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,240 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,240 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,241 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:54:56,241 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:54:56,241 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:56,490 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,490 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:54:56,490 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:54:56,490 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:54:56,490 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-07-24 07:54:56,490 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:54:56,490 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:54:56,490 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:54:56,490 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:54:56,491 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:54:56,492 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-24 07:54:56,493 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:54:56,493 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:54:56,493 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:54:56,493 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:54:56,493 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:56,494 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:54:56,494 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:56,494 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:56,494 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:56,494 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:54:56,494 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:56,494 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:56,495 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:56,495 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:54:56,495 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:54:56,495 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:56,495 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:56,495 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:56,495 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:54:56,496 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:56,496 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:56,496 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:56,496 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:54:56,496 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:56,496 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:56,497 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:56,497 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:56,497 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:56,498 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:56,498 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:56,498 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:56,498 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:56,498 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:56,499 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:56,499 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,499 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:54:56,499 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,499 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,499 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,500 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-24 07:54:56,500 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:54:56,501 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:54:56,501 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185326501 2023-07-24 07:54:56,502 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:54:56,503 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,505 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:54:56,505 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=63edc6e611d2d72dc562948ea214a40d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:54:56,505 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:54:56,505 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185356505 2023-07-24 07:54:56,505 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185416505 2023-07-24 07:54:56,505 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:54:56,506 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:54:56,507 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:54:56,508 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:54:56,508 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:56,756 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,756 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:54:56,756 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:54:56,756 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:54:56,756 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:54:56,756 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:54:56,756 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:54:56,756 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:54:56,756 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:54:56,757 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:54:56,758 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-24 07:54:56,759 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:54:56,759 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:54:56,759 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:54:56,760 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:54:56,760 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:56,760 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:54:56,760 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:56,761 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:56,761 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:56,761 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:54:56,761 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:56,761 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:56,762 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:56,762 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:54:56,762 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:54:56,762 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:56,762 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:56,763 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:56,763 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:54:56,763 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:56,763 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:56,763 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:56,763 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:54:56,763 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:56,763 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:56,764 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:56,764 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:56,764 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:56,764 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:56,764 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:56,764 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:56,764 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:56,765 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:56,765 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:56,765 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,765 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:54:56,765 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,765 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,765 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,766 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-24 07:54:56,766 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:54:56,767 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:54:56,767 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185326767 2023-07-24 07:54:56,769 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:54:56,769 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,771 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:54:56,771 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=63edc6e611d2d72dc562948ea214a40d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:54:56,771 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:54:56,771 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185356771 2023-07-24 07:54:56,771 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185416771 2023-07-24 07:54:56,771 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 07:54:56,771 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:54:56,773 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN}] 2023-07-24 07:54:56,773 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:54:56,773 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:56,773 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:54:56,773 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:57,021 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,022 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:54:57,022 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:54:57,022 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:54:57,023 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:54:57,024 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:54:57,024 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:54:57,024 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:54:57,024 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:54:57,025 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:54:57,027 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-24 07:54:57,027 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:54:57,028 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:54:57,028 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:54:57,028 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:54:57,029 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:57,029 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:54:57,029 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:57,029 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:57,029 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:57,029 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:54:57,030 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:57,030 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:57,030 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:57,030 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:54:57,030 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:54:57,030 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:57,031 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:57,031 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:57,031 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:54:57,031 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:57,031 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:57,031 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:57,031 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:54:57,031 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:57,031 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:57,032 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,032 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,032 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:57,032 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,032 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,032 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:57,033 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:57,033 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:57,033 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:57,033 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,033 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:54:57,034 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,034 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,034 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,034 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-24 07:54:57,035 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:54:57,036 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:54:57,036 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185327036 2023-07-24 07:54:57,037 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:54:57,039 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,041 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:54:57,041 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=63edc6e611d2d72dc562948ea214a40d, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:54:57,041 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:54:57,041 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185357041 2023-07-24 07:54:57,041 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185417041 2023-07-24 07:54:57,042 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 07:54:57,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN}] 2023-07-24 07:54:57,042 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:54:57,044 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN 2023-07-24 07:54:57,045 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=63edc6e611d2d72dc562948ea214a40d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:57,045 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185297045"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185297045"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185297045"}]},"ts":"1690185297045"} 2023-07-24 07:54:57,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:54:57,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 63edc6e611d2d72dc562948ea214a40d, disabling compactions & flushes 2023-07-24 07:54:57,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:57,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:57,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. after waiting 0 ms 2023-07-24 07:54:57,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:57,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 63edc6e611d2d72dc562948ea214a40d 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 07:54:57,208 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/4226e054714c4683952791276791f341 2023-07-24 07:54:57,219 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/f759ed834f994113bdfb5ee3f8566c56 2023-07-24 07:54:57,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf1/4226e054714c4683952791276791f341 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341 2023-07-24 07:54:57,226 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:54:57,227 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/.tmp/cf2/f759ed834f994113bdfb5ee3f8566c56 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56 2023-07-24 07:54:57,230 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:54:57,230 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 63edc6e611d2d72dc562948ea214a40d in 32ms, sequenceid=82, compaction requested=true 2023-07-24 07:54:57,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 07:54:57,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:54:57,238 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. 2023-07-24 07:54:57,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 63edc6e611d2d72dc562948ea214a40d: 2023-07-24 07:54:57,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,240 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=63edc6e611d2d72dc562948ea214a40d, regionState=CLOSED 2023-07-24 07:54:57,240 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185297240"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185297240"}]},"ts":"1690185297240"} 2023-07-24 07:54:57,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-07-24 07:54:57,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 in 195 msec 2023-07-24 07:54:57,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-07-24 07:54:57,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN in 201 msec 2023-07-24 07:54:57,246 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:54:57,246 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,246 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:54:57,246 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:57,292 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,292 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:54:57,292 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:54:57,292 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:54:57,292 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:54:57,292 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:54:57,292 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:54:57,292 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:54:57,293 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:54:57,293 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:54:57,294 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 07:54:57,294 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:54:57,294 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:54:57,295 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:54:57,295 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:54:57,295 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:54:57,295 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:54:57,295 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:57,295 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:54:57,296 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:57,296 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:57,296 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:57,296 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:54:57,296 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:57,296 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:57,296 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:57,297 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:54:57,297 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:54:57,297 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:57,297 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:57,297 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:57,297 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:54:57,298 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:57,298 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:57,298 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:57,298 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:54:57,298 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:57,298 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:57,298 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,299 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,299 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:57,299 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,299 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,300 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:57,300 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:57,300 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:57,300 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:57,300 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,301 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN 2023-07-24 07:54:57,301 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:57,301 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:54:57,301 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,301 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,301 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,301 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:57,301 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:57,301 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-24 07:54:57,302 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:54:57,302 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:54:57,302 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185327302 2023-07-24 07:54:57,303 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:54:57,304 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,307 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:54:57,307 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=63edc6e611d2d72dc562948ea214a40d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:54:57,307 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:54:57,307 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185357307 2023-07-24 07:54:57,307 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185417307 2023-07-24 07:54:57,307 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 07:54:57,310 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:54:57,319 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=63edc6e611d2d72dc562948ea214a40d, threads=14 2023-07-24 07:54:57,320 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,320 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,320 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,320 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,320 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,320 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,320 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,320 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,320 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,321 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,321 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,321 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,321 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,321 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,504 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,506 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,513 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,523 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,523 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,527 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,534 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,539 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,544 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,544 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,548 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 07:54:57,549 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,562 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,570 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,572 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,572 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 63edc6e611d2d72dc562948ea214a40d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 07:54:57,607 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:54:57,608 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:54:57,608 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:57,808 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,808 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:54:57,808 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:54:57,808 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:54:57,809 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:54:57,810 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:54:57,810 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:54:57,810 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:54:57,810 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:54:57,811 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:54:57,812 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-24 07:54:57,813 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:54:57,813 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:54:57,813 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:54:57,813 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:54:57,813 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:54:57,814 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:54:57,814 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:57,814 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:54:57,814 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:57,815 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:57,815 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:57,815 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:54:57,815 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:57,815 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:57,816 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:57,816 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:54:57,816 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:54:57,816 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:57,817 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:57,817 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:57,817 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:54:57,817 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:57,817 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:57,818 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:57,818 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:54:57,818 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:57,818 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:57,818 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,819 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,819 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:57,819 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,819 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:57,820 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:57,820 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:57,820 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:57,820 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:57,820 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,821 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN 2023-07-24 07:54:57,821 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:57,821 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:54:57,821 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,821 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,821 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,821 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:57,821 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:57,822 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-24 07:54:57,822 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:54:57,823 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:54:57,823 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185327823 2023-07-24 07:54:57,825 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:57,825 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:54:57,827 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:54:57,828 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=63edc6e611d2d72dc562948ea214a40d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:54:57,828 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:54:57,828 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185357828 2023-07-24 07:54:57,828 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185417828 2023-07-24 07:54:57,828 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:54:57,836 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=63edc6e611d2d72dc562948ea214a40d, threads=14 2023-07-24 07:54:57,837 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,837 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,837 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,837 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,839 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,839 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,839 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,839 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,839 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,840 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,840 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,842 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,843 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,843 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,991 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:57,995 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,001 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,023 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,027 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,027 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,027 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,039 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,040 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,041 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,041 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,043 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,044 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,044 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3 for region: 63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,044 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 63edc6e611d2d72dc562948ea214a40d Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 07:54:58,077 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:54:58,087 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:54:58,088 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:54:58,088 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:54:58,088 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:54:58,088 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:58,328 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,329 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:54:58,329 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:54:58,329 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:54:58,330 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:54:58,330 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:54:58,330 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:54:58,330 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:54:58,331 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:54:58,331 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:54:58,332 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-24 07:54:58,332 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:54:58,333 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:54:58,333 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:54:58,333 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:54:58,333 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:54:58,333 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:54:58,334 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:58,334 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:54:58,334 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:58,334 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:58,334 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:58,334 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:54:58,335 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:58,335 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:58,335 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:58,335 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:54:58,335 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:54:58,335 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:58,335 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:58,336 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:58,336 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:54:58,336 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:58,336 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:58,336 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:58,336 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:54:58,336 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:58,336 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:58,336 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,337 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,337 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:58,337 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,337 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,337 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:58,337 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:58,337 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:58,338 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:58,338 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,338 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN 2023-07-24 07:54:58,338 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:58,338 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:54:58,338 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,338 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,338 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,338 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:58,338 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:58,338 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-24 07:54:58,339 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:54:58,340 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:54:58,340 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185328340 2023-07-24 07:54:58,341 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:54:58,341 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,343 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:54:58,344 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=63edc6e611d2d72dc562948ea214a40d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:54:58,344 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:54:58,344 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185358344 2023-07-24 07:54:58,344 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185418344 2023-07-24 07:54:58,344 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 07:54:58,347 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:54:58,348 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:54:58,349 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,349 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:54:58,349 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:58,594 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,594 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:54:58,594 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:54:58,595 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:54:58,596 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:54:58,596 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:54:58,596 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:54:58,596 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:54:58,596 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:54:58,597 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:54:58,598 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-24 07:54:58,598 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:54:58,599 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:54:58,599 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:54:58,599 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:54:58,599 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:54:58,599 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:54:58,599 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:58,600 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:54:58,600 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:58,600 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:58,600 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:58,600 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:54:58,600 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:58,600 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:58,600 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:58,601 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:54:58,601 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:54:58,601 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:58,601 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:58,601 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:58,601 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:54:58,601 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:58,601 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:58,601 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:58,602 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:54:58,602 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:58,602 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:58,602 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,602 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,602 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:58,602 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,602 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,603 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:58,603 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:58,603 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:58,603 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:58,603 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,604 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN 2023-07-24 07:54:58,604 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:58,604 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:54:58,604 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,604 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,604 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,604 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:58,604 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:58,604 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-24 07:54:58,605 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:54:58,606 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:54:58,606 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185328606 2023-07-24 07:54:58,608 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:54:58,608 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,609 ERROR [BP-80517894-172.31.14.131-1690185194253 heartbeating to localhost/127.0.0.1:34211] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-80517894-172.31.14.131-1690185194253 (Datanode Uuid b4241ba3-003c-4dbd-8dcc-7026b912408a) service to localhost/127.0.0.1:34211 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-24 07:54:58,611 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:54:58,611 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=63edc6e611d2d72dc562948ea214a40d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:54:58,611 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:54:58,611 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185358611 2023-07-24 07:54:58,611 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185418611 2023-07-24 07:54:58,611 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:54:58,612 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:54:58,614 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185298614"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1690185298614"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1690185298614"}]},"ts":"1690185298614"} 2023-07-24 07:54:58,614 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185298614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185298614"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185298614"}]},"ts":"1690185298614"} 2023-07-24 07:54:58,614 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185298614"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185298614"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185298614"}]},"ts":"1690185298614"} 2023-07-24 07:54:58,617 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:54:58,617 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:54:58,617 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:58,862 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,862 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:54:58,862 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:54:58,862 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:54:58,862 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:54:58,862 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:54:58,862 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:54:58,862 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:54:58,862 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:54:58,863 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:54:58,864 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-07-24 07:54:58,865 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:54:58,865 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:54:58,865 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:54:58,865 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:54:58,866 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:54:58,866 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:54:58,866 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:58,866 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:54:58,867 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:58,867 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:58,867 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:58,867 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:54:58,867 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:58,868 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:58,868 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:58,868 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:54:58,869 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:54:58,869 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:58,869 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:58,869 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:58,870 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:54:58,870 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:58,870 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:58,870 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:58,871 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:54:58,871 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:58,871 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:58,871 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,872 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,872 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:58,872 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,872 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:58,873 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:58,873 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:58,873 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:58,873 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:58,874 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,874 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN 2023-07-24 07:54:58,874 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:58,874 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:54:58,874 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,874 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,874 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,875 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:58,875 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:58,875 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-24 07:54:58,875 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:54:58,876 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:54:58,876 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185328876 2023-07-24 07:54:58,878 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,878 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:54:58,880 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:54:58,880 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=63edc6e611d2d72dc562948ea214a40d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:54:58,881 INFO [Listener at localhost/42537] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:58,881 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=488ee2c34de56e3764da09c2c2def106, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 07:54:58,881 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=2123d27523d3bca109f37599c76acc9a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 07:54:58,881 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:54:58,881 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185358881 2023-07-24 07:54:58,881 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185418881 2023-07-24 07:54:58,881 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:54:58,882 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690185298882"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1690185298882"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1690185298882"}]},"ts":"1690185298882"} 2023-07-24 07:54:58,882 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185298882"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185298882"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185298882"}]},"ts":"1690185298882"} 2023-07-24 07:54:58,882 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185298882"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185298882"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185298882"}]},"ts":"1690185298882"} 2023-07-24 07:54:58,884 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:54:58,886 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:54:58,886 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:58,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:54:58,886 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:59,132 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,132 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:54:59,132 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:54:59,132 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:54:59,132 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:54:59,132 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:54:59,132 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:54:59,132 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:54:59,132 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:54:59,133 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:54:59,134 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-07-24 07:54:59,134 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:54:59,134 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:54:59,134 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:54:59,135 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:54:59,135 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:54:59,135 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:54:59,135 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:59,135 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:54:59,135 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:59,135 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:59,136 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:59,136 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:54:59,136 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:59,136 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:59,136 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:59,136 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:54:59,136 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:54:59,137 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:59,137 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:59,137 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:59,137 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:54:59,137 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:59,137 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:59,137 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:59,137 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:54:59,137 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:59,138 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:59,138 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,138 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,138 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:59,138 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,138 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,139 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:59,139 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:59,139 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:59,139 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:59,139 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,139 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN 2023-07-24 07:54:59,139 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:59,139 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:54:59,139 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,140 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,140 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,140 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:59,140 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:59,140 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 07:54:59,140 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:54:59,141 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:54:59,141 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185329141 2023-07-24 07:54:59,143 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,143 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:54:59,145 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:54:59,145 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=63edc6e611d2d72dc562948ea214a40d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:54:59,145 INFO [Listener at localhost/42537] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:59,145 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=488ee2c34de56e3764da09c2c2def106, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 07:54:59,146 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=2123d27523d3bca109f37599c76acc9a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 07:54:59,146 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:54:59,146 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185359146 2023-07-24 07:54:59,146 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185419146 2023-07-24 07:54:59,146 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 07:54:59,146 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:54:59,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=488ee2c34de56e3764da09c2c2def106, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2123d27523d3bca109f37599c76acc9a, ASSIGN}] 2023-07-24 07:54:59,148 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:54:59,148 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:54:59,148 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:59,396 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,396 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:54:59,396 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:54:59,396 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:54:59,397 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:54:59,397 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:54:59,397 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:54:59,397 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:54:59,397 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:54:59,397 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:54:59,399 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-07-24 07:54:59,399 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:54:59,399 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:54:59,399 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:54:59,399 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:54:59,399 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:54:59,400 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:54:59,400 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:59,400 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:54:59,400 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:59,400 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:59,400 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:59,400 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:54:59,400 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:59,401 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:59,401 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:59,401 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:54:59,401 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:54:59,401 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:59,401 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:59,401 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:59,401 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:54:59,402 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:59,402 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:59,402 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:59,402 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:54:59,402 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:59,403 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:59,403 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,403 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,403 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:59,404 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,404 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,404 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:59,404 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:59,404 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:59,405 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:59,405 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,405 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN 2023-07-24 07:54:59,405 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:59,405 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:54:59,406 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,406 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,406 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,406 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:59,406 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:59,406 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-24 07:54:59,407 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:54:59,408 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:54:59,409 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185329409 2023-07-24 07:54:59,411 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:54:59,411 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,414 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:54:59,414 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=63edc6e611d2d72dc562948ea214a40d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:54:59,414 INFO [Listener at localhost/42537] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:59,414 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=488ee2c34de56e3764da09c2c2def106, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 07:54:59,414 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=2123d27523d3bca109f37599c76acc9a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 07:54:59,414 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:54:59,415 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185359415 2023-07-24 07:54:59,415 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185419415 2023-07-24 07:54:59,415 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 07:54:59,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=488ee2c34de56e3764da09c2c2def106, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2123d27523d3bca109f37599c76acc9a, ASSIGN}] 2023-07-24 07:54:59,415 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:54:59,417 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2123d27523d3bca109f37599c76acc9a, ASSIGN 2023-07-24 07:54:59,417 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2123d27523d3bca109f37599c76acc9a, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:54:59,418 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=488ee2c34de56e3764da09c2c2def106, ASSIGN 2023-07-24 07:54:59,419 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=488ee2c34de56e3764da09c2c2def106, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:54:59,568 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 07:54:59,569 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=488ee2c34de56e3764da09c2c2def106, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:59,570 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185299569"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185299569"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185299569"}]},"ts":"1690185299569"} 2023-07-24 07:54:59,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 488ee2c34de56e3764da09c2c2def106, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:54:59,573 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=2123d27523d3bca109f37599c76acc9a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:59,573 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185299573"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185299573"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185299573"}]},"ts":"1690185299573"} 2023-07-24 07:54:59,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 2123d27523d3bca109f37599c76acc9a, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:54:59,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. 2023-07-24 07:54:59,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 488ee2c34de56e3764da09c2c2def106, NAME => 'testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.', STARTKEY => '', ENDKEY => '41'} 2023-07-24 07:54:59,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:54:59,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 488ee2c34de56e3764da09c2c2def106 2023-07-24 07:54:59,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:54:59,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 488ee2c34de56e3764da09c2c2def106 2023-07-24 07:54:59,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 488ee2c34de56e3764da09c2c2def106 2023-07-24 07:54:59,727 INFO [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 488ee2c34de56e3764da09c2c2def106 2023-07-24 07:54:59,728 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1 2023-07-24 07:54:59,728 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1 2023-07-24 07:54:59,728 INFO [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 488ee2c34de56e3764da09c2c2def106 columnFamilyName cf1 2023-07-24 07:54:59,740 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e-bottom 2023-07-24 07:54:59,744 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3-bottom 2023-07-24 07:54:59,747 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84-bottom 2023-07-24 07:54:59,750 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2-bottom 2023-07-24 07:54:59,754 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341-bottom 2023-07-24 07:54:59,758 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5-bottom 2023-07-24 07:54:59,762 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59-bottom 2023-07-24 07:54:59,762 INFO [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(310): Store=488ee2c34de56e3764da09c2c2def106/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:54:59,762 INFO [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 488ee2c34de56e3764da09c2c2def106 2023-07-24 07:54:59,763 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2 2023-07-24 07:54:59,763 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2 2023-07-24 07:54:59,763 INFO [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 488ee2c34de56e3764da09c2c2def106 columnFamilyName cf2 2023-07-24 07:54:59,776 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627-bottom 2023-07-24 07:54:59,779 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb-bottom 2023-07-24 07:54:59,783 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0-bottom 2023-07-24 07:54:59,787 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277-bottom 2023-07-24 07:54:59,790 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48-bottom 2023-07-24 07:54:59,795 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f-bottom 2023-07-24 07:54:59,799 DEBUG [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56-bottom 2023-07-24 07:54:59,799 INFO [StoreOpener-488ee2c34de56e3764da09c2c2def106-1] regionserver.HStore(310): Store=488ee2c34de56e3764da09c2c2def106/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:54:59,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106 2023-07-24 07:54:59,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106 2023-07-24 07:54:59,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:54:59,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 488ee2c34de56e3764da09c2c2def106 2023-07-24 07:54:59,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 488ee2c34de56e3764da09c2c2def106; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61502676, jitterRate=-0.0835387110710144}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:54:59,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 488ee2c34de56e3764da09c2c2def106: 2023-07-24 07:54:59,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106., pid=117, masterSystemTime=1690185299722 2023-07-24 07:54:59,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:54:59,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 07:54:59,808 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:54:59,809 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:54:59,809 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. 2023-07-24 07:54:59,809 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1912): 488ee2c34de56e3764da09c2c2def106/cf1 is initiating minor compaction (all files) 2023-07-24 07:54:59,809 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. 2023-07-24 07:54:59,809 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1912): 488ee2c34de56e3764da09c2c2def106/cf2 is initiating minor compaction (all files) 2023-07-24 07:54:59,809 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 488ee2c34de56e3764da09c2c2def106/cf1 in testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. 2023-07-24 07:54:59,809 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 488ee2c34de56e3764da09c2c2def106/cf2 in testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. 2023-07-24 07:54:59,809 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341-bottom] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/.tmp, totalSize=34.9 K 2023-07-24 07:54:59,809 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56-bottom] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/.tmp, totalSize=34.9 K 2023-07-24 07:54:59,810 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185294885 2023-07-24 07:54:59,810 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185294885 2023-07-24 07:54:59,810 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185295057 2023-07-24 07:54:59,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. 2023-07-24 07:54:59,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. 2023-07-24 07:54:59,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. 2023-07-24 07:54:59,810 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=488ee2c34de56e3764da09c2c2def106, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:59,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2123d27523d3bca109f37599c76acc9a, NAME => 'testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.', STARTKEY => '41', ENDKEY => ''} 2023-07-24 07:54:59,810 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185299810"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185299810"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185299810"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185299810"}]},"ts":"1690185299810"} 2023-07-24 07:54:59,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:54:59,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:54:59,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,811 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185295057 2023-07-24 07:54:59,811 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185295259 2023-07-24 07:54:59,812 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185295259 2023-07-24 07:54:59,812 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185295432 2023-07-24 07:54:59,812 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185295432 2023-07-24 07:54:59,812 INFO [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,812 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185295604 2023-07-24 07:54:59,813 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185295604 2023-07-24 07:54:59,813 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185295802 2023-07-24 07:54:59,813 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185295802 2023-07-24 07:54:59,813 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1 2023-07-24 07:54:59,813 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1 2023-07-24 07:54:59,813 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185295981 2023-07-24 07:54:59,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-07-24 07:54:59,814 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185295981 2023-07-24 07:54:59,814 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 488ee2c34de56e3764da09c2c2def106, server=jenkins-hbase4.apache.org,46543,1690185199091 in 241 msec 2023-07-24 07:54:59,814 INFO [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2123d27523d3bca109f37599c76acc9a columnFamilyName cf1 2023-07-24 07:54:59,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=488ee2c34de56e3764da09c2c2def106, ASSIGN in 399 msec 2023-07-24 07:54:59,825 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] throttle.PressureAwareThroughputController(145): 488ee2c34de56e3764da09c2c2def106#cf1#compaction#106 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:54:59,825 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] throttle.PressureAwareThroughputController(145): 488ee2c34de56e3764da09c2c2def106#cf2#compaction#107 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:54:59,834 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e-top 2023-07-24 07:54:59,837 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3-top 2023-07-24 07:54:59,840 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84-top 2023-07-24 07:54:59,844 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2-top 2023-07-24 07:54:59,848 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/.tmp/cf1/bc4a948fc4cc46c38ef007c434d7c046 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/bc4a948fc4cc46c38ef007c434d7c046 2023-07-24 07:54:59,848 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/.tmp/cf2/903d14b9d1e44e16be51786de47d5416 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/903d14b9d1e44e16be51786de47d5416 2023-07-24 07:54:59,849 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341-top 2023-07-24 07:54:59,853 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 488ee2c34de56e3764da09c2c2def106/cf1 of 488ee2c34de56e3764da09c2c2def106 into bc4a948fc4cc46c38ef007c434d7c046(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:54:59,853 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 488ee2c34de56e3764da09c2c2def106: 2023-07-24 07:54:59,853 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106., storeName=488ee2c34de56e3764da09c2c2def106/cf1, priority=9, startTime=1690185299805; duration=0sec 2023-07-24 07:54:59,854 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:54:59,854 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 488ee2c34de56e3764da09c2c2def106/cf2 of 488ee2c34de56e3764da09c2c2def106 into 903d14b9d1e44e16be51786de47d5416(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:54:59,854 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 488ee2c34de56e3764da09c2c2def106: 2023-07-24 07:54:59,854 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106., storeName=488ee2c34de56e3764da09c2c2def106/cf2, priority=9, startTime=1690185299808; duration=0sec 2023-07-24 07:54:59,854 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5-top 2023-07-24 07:54:59,854 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:54:59,858 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59-top 2023-07-24 07:54:59,858 INFO [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(310): Store=2123d27523d3bca109f37599c76acc9a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:54:59,858 INFO [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,859 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2 2023-07-24 07:54:59,859 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2 2023-07-24 07:54:59,859 INFO [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2123d27523d3bca109f37599c76acc9a columnFamilyName cf2 2023-07-24 07:54:59,870 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627-top 2023-07-24 07:54:59,874 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb-top 2023-07-24 07:54:59,877 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0-top 2023-07-24 07:54:59,881 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277-top 2023-07-24 07:54:59,884 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48-top 2023-07-24 07:54:59,887 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f-top 2023-07-24 07:54:59,891 DEBUG [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56-top 2023-07-24 07:54:59,891 INFO [StoreOpener-2123d27523d3bca109f37599c76acc9a-1] regionserver.HStore(310): Store=2123d27523d3bca109f37599c76acc9a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:54:59,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,892 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:54:59,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2123d27523d3bca109f37599c76acc9a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62312584, jitterRate=-0.07147014141082764}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:54:59,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2123d27523d3bca109f37599c76acc9a: 2023-07-24 07:54:59,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a., pid=118, masterSystemTime=1690185299722 2023-07-24 07:54:59,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:54:59,898 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:54:59,898 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:54:59,898 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:54:59,898 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. 2023-07-24 07:54:59,898 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1912): 2123d27523d3bca109f37599c76acc9a/cf1 is initiating minor compaction (all files) 2023-07-24 07:54:59,898 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. 2023-07-24 07:54:59,898 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1912): 2123d27523d3bca109f37599c76acc9a/cf2 is initiating minor compaction (all files) 2023-07-24 07:54:59,898 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2123d27523d3bca109f37599c76acc9a/cf1 in testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. 2023-07-24 07:54:59,898 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 2123d27523d3bca109f37599c76acc9a/cf2 in testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. 2023-07-24 07:54:59,898 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56-top] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/.tmp, totalSize=34.9 K 2023-07-24 07:54:59,898 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341-top] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/.tmp, totalSize=34.9 K 2023-07-24 07:54:59,899 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690185294885 2023-07-24 07:54:59,899 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690185294885 2023-07-24 07:54:59,899 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690185295057 2023-07-24 07:54:59,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. 2023-07-24 07:54:59,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. 2023-07-24 07:54:59,900 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690185295057 2023-07-24 07:54:59,900 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690185295259 2023-07-24 07:54:59,900 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=2123d27523d3bca109f37599c76acc9a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:59,900 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185299900"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185299900"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185299900"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185299900"}]},"ts":"1690185299900"} 2023-07-24 07:54:59,900 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690185295259 2023-07-24 07:54:59,900 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690185295432 2023-07-24 07:54:59,901 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690185295432 2023-07-24 07:54:59,901 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690185295604 2023-07-24 07:54:59,901 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690185295802 2023-07-24 07:54:59,901 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690185295604 2023-07-24 07:54:59,902 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690185295981 2023-07-24 07:54:59,902 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690185295802 2023-07-24 07:54:59,902 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690185295981 2023-07-24 07:54:59,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-07-24 07:54:59,903 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 2123d27523d3bca109f37599c76acc9a, server=jenkins-hbase4.apache.org,46543,1690185199091 in 327 msec 2023-07-24 07:54:59,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-07-24 07:54:59,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2123d27523d3bca109f37599c76acc9a, ASSIGN in 488 msec 2023-07-24 07:54:59,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 07:54:59,905 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:54:59,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 07:54:59,915 INFO [Listener at localhost/42537] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,915 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 07:54:59,915 INFO [Listener at localhost/42537] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:54:59,915 INFO [Listener at localhost/42537] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:54:59,916 ERROR [Listener at localhost/42537] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 07:54:59,917 INFO [Listener at localhost/42537] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 07:54:59,917 INFO [Listener at localhost/42537] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 07:54:59,917 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 07:54:59,917 INFO [Listener at localhost/42537] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 07:54:59,917 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 07:54:59,918 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-07-24 07:54:59,918 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 07:54:59,918 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 07:54:59,918 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-07-24 07:54:59,919 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-07-24 07:54:59,919 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-07-24 07:54:59,919 DEBUG [Listener at localhost/42537] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-07-24 07:54:59,919 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 07:54:59,919 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 07:54:59,919 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 07:54:59,919 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 07:54:59,919 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:59,920 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=a9f3298786bf73c446662e1bb906c6e5, daughterA=5eca7056352c3c5c9619dde691b5b9b3, daughterB=542548d310d66672741e6f9e560f852c 2023-07-24 07:54:59,920 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:59,920 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 07:54:59,920 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:59,920 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=7005ae1188bcf4c865415273ba071945, daughterA=efa5009ac14fc821e717f05c40ca0ce6, daughterB=cee8e570880880b54e8e6fcbd087cb82 2023-07-24 07:54:59,920 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:59,920 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 07:54:59,921 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:59,921 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=3d3705bd1d338a8a463a7f16f68aba1b, daughterA=5a709f00c41c4ec47f7254db00f24be6, daughterB=9ab9b0c2f983da01540f56de416e8725 2023-07-24 07:54:59,921 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=3d3705bd1d338a8a463a7f16f68aba1b, ASSIGN 2023-07-24 07:54:59,921 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:59,921 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 07:54:59,921 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:59,921 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=df88344c1a5d2a50942a8f97222251c3, daughterA=950a2f2cad20e0dd1edc0ff5cb3fe097, daughterB=47fb575219491f8a6dfebbe5390ced75 2023-07-24 07:54:59,921 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:59,921 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 07:54:59,922 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:59,922 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=d0b3800d667beee4d07b06d26530b356, daughterA=bce71ec85c4b67fcb754f132e5380e54, daughterB=d31cb3be64e26e69f92b7e2238413984 2023-07-24 07:54:59,922 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:59,922 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 07:54:59,922 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,922 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,922 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRollbackForSplitTableRegionWithReplica, region=a0ab33d4ecb05d862ed118a501d92b50 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0be6421af829d9242b43814b95c3339d, daughterA=07b67580fd5cc2e2c6d15b2b1c2ec3e3, daughterB=d1b2090fd6228f010dfb8a2d53b22f6c 2023-07-24 07:54:59,923 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,923 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 07:54:59,923 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:59,923 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:59,923 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 07:54:59,924 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:54:59,925 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,925 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2123d27523d3bca109f37599c76acc9a#cf1#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:54:59,925 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN 2023-07-24 07:54:59,925 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:59,925 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=488ee2c34de56e3764da09c2c2def106, ASSIGN 2023-07-24 07:54:59,925 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2123d27523d3bca109f37599c76acc9a, ASSIGN 2023-07-24 07:54:59,925 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 488ee2c34de56e3764da09c2c2def106, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:59,925 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 2123d27523d3bca109f37599c76acc9a, server=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:54:59,926 DEBUG [Listener at localhost/42537] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a held the lock before restarting, call acquireLock to restore it. 2023-07-24 07:54:59,926 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,926 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,926 INFO [Listener at localhost/42537] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,926 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:59,926 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 63edc6e611d2d72dc562948ea214a40d, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:59,926 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=488ee2c34de56e3764da09c2c2def106, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:59,926 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2123d27523d3bca109f37599c76acc9a, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:59,926 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 488ee2c34de56e3764da09c2c2def106, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:59,926 DEBUG [Listener at localhost/42537] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 2123d27523d3bca109f37599c76acc9a, server=jenkins-hbase4.apache.org,46543,1690185199091 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 07:54:59,927 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-07-24 07:54:59,927 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] throttle.PressureAwareThroughputController(145): 2123d27523d3bca109f37599c76acc9a#cf2#compaction#109 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:54:59,927 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 07:54:59,928 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37809,1690185199136, table=hbase:meta, region=1588230740 2023-07-24 07:54:59,936 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690185329935 2023-07-24 07:54:59,936 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a 2023-07-24 07:54:59,936 DEBUG [Listener at localhost/42537] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 07:54:59,940 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=61dcffa18562b861467de5a803476e2c, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37809,1690185199136, regionLocation=jenkins-hbase4.apache.org,37809,1690185199136, openSeqNum=2 2023-07-24 07:54:59,940 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=63edc6e611d2d72dc562948ea214a40d, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=2 2023-07-24 07:54:59,940 INFO [Listener at localhost/42537] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,46543,1690185199091, table=testRecoveryAndDoubleExecution, region=63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:54:59,940 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=488ee2c34de56e3764da09c2c2def106, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=86 2023-07-24 07:54:59,940 DEBUG [Listener at localhost/42537] assignment.RegionStateStore(147): Load hbase:meta entry region=2123d27523d3bca109f37599c76acc9a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46543,1690185199091, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091, openSeqNum=86 2023-07-24 07:54:59,940 INFO [Listener at localhost/42537] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 07:54:59,940 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690185359940 2023-07-24 07:54:59,941 INFO [Listener at localhost/42537] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690185419941 2023-07-24 07:54:59,941 INFO [Listener at localhost/42537] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 07:54:59,947 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 07:54:59,951 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=63edc6e611d2d72dc562948ea214a40d, daughterA=488ee2c34de56e3764da09c2c2def106, daughterB=2123d27523d3bca109f37599c76acc9a in 3.7080 sec 2023-07-24 07:54:59,957 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/.tmp/cf2/b4cd007ae7fe458b969e7b28d9f3bd50 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/b4cd007ae7fe458b969e7b28d9f3bd50 2023-07-24 07:54:59,961 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2123d27523d3bca109f37599c76acc9a/cf2 of 2123d27523d3bca109f37599c76acc9a into b4cd007ae7fe458b969e7b28d9f3bd50(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:54:59,961 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2123d27523d3bca109f37599c76acc9a: 2023-07-24 07:54:59,961 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a., storeName=2123d27523d3bca109f37599c76acc9a/cf2, priority=9, startTime=1690185299898; duration=0sec 2023-07-24 07:54:59,961 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:00,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46543] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:36420 deadline: 1690185360198, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d. is not online on jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:00,311 DEBUG [Listener at localhost/42537] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 07:55:00,317 INFO [Listener at localhost/42537] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-24 07:55:00,319 DEBUG [Listener at localhost/42537] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 07:55:00,334 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:55:00,336 INFO [Listener at localhost/42537] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-07-24 07:55:00,336 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-07-24 07:55:00,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:55:00,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-24 07:55:00,339 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185300339"}]},"ts":"1690185300339"} 2023-07-24 07:55:00,340 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-07-24 07:55:00,341 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-07-24 07:55:00,342 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=488ee2c34de56e3764da09c2c2def106, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2123d27523d3bca109f37599c76acc9a, UNASSIGN}] 2023-07-24 07:55:00,343 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2123d27523d3bca109f37599c76acc9a, UNASSIGN 2023-07-24 07:55:00,343 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=2123d27523d3bca109f37599c76acc9a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:00,343 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185300343"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185300343"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185300343"}]},"ts":"1690185300343"} 2023-07-24 07:55:00,344 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 2123d27523d3bca109f37599c76acc9a, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:55:00,347 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=488ee2c34de56e3764da09c2c2def106, UNASSIGN 2023-07-24 07:55:00,347 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=488ee2c34de56e3764da09c2c2def106, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:00,347 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185300347"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185300347"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185300347"}]},"ts":"1690185300347"} 2023-07-24 07:55:00,348 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 488ee2c34de56e3764da09c2c2def106, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:55:00,350 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/.tmp/cf1/7b9b3680e6c54abe97585e941f6d5624 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/7b9b3680e6c54abe97585e941f6d5624 2023-07-24 07:55:00,354 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2123d27523d3bca109f37599c76acc9a/cf1 of 2123d27523d3bca109f37599c76acc9a into 7b9b3680e6c54abe97585e941f6d5624(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:55:00,354 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2123d27523d3bca109f37599c76acc9a: 2023-07-24 07:55:00,354 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a., storeName=2123d27523d3bca109f37599c76acc9a/cf1, priority=9, startTime=1690185299897; duration=0sec 2023-07-24 07:55:00,355 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:00,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-24 07:55:00,496 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 488ee2c34de56e3764da09c2c2def106 2023-07-24 07:55:00,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 488ee2c34de56e3764da09c2c2def106, disabling compactions & flushes 2023-07-24 07:55:00,497 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. 2023-07-24 07:55:00,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. 2023-07-24 07:55:00,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. after waiting 0 ms 2023-07-24 07:55:00,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. 2023-07-24 07:55:00,497 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341-bottom] to archive 2023-07-24 07:55:00,498 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:55:00,500 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,501 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,502 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,503 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,504 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,505 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,506 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,508 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56-bottom] to archive 2023-07-24 07:55:00,508 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:55:00,509 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,511 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,511 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,512 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,513 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,515 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,516 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,519 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 07:55:00,519 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:55:00,519 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106. 2023-07-24 07:55:00,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 488ee2c34de56e3764da09c2c2def106: 2023-07-24 07:55:00,522 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 488ee2c34de56e3764da09c2c2def106 2023-07-24 07:55:00,522 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2123d27523d3bca109f37599c76acc9a 2023-07-24 07:55:00,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2123d27523d3bca109f37599c76acc9a, disabling compactions & flushes 2023-07-24 07:55:00,522 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. 2023-07-24 07:55:00,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. 2023-07-24 07:55:00,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. after waiting 0 ms 2023-07-24 07:55:00,522 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=488ee2c34de56e3764da09c2c2def106, regionState=CLOSED 2023-07-24 07:55:00,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. 2023-07-24 07:55:00,522 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185300522"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185300522"}]},"ts":"1690185300522"} 2023-07-24 07:55:00,523 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341-top] to archive 2023-07-24 07:55:00,524 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:55:00,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-07-24 07:55:00,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 488ee2c34de56e3764da09c2c2def106, server=jenkins-hbase4.apache.org,46543,1690185199091 in 176 msec 2023-07-24 07:55:00,526 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=488ee2c34de56e3764da09c2c2def106, UNASSIGN in 183 msec 2023-07-24 07:55:00,527 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,528 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,529 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,530 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,531 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,532 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,533 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56-top] to archive 2023-07-24 07:55:00,534 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:55:00,535 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,536 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,537 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,538 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,539 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,540 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,541 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 07:55:00,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:55:00,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a. 2023-07-24 07:55:00,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2123d27523d3bca109f37599c76acc9a: 2023-07-24 07:55:00,547 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2123d27523d3bca109f37599c76acc9a 2023-07-24 07:55:00,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=2123d27523d3bca109f37599c76acc9a, regionState=CLOSED 2023-07-24 07:55:00,548 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690185300548"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185300548"}]},"ts":"1690185300548"} 2023-07-24 07:55:00,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-07-24 07:55:00,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 2123d27523d3bca109f37599c76acc9a, server=jenkins-hbase4.apache.org,46543,1690185199091 in 205 msec 2023-07-24 07:55:00,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-07-24 07:55:00,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=2123d27523d3bca109f37599c76acc9a, UNASSIGN in 208 msec 2023-07-24 07:55:00,552 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185300552"}]},"ts":"1690185300552"} 2023-07-24 07:55:00,553 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-07-24 07:55:00,555 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-07-24 07:55:00,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 219 msec 2023-07-24 07:55:00,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-24 07:55:00,641 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-07-24 07:55:00,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-07-24 07:55:00,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:55:00,644 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:55:00,645 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:55:00,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-07-24 07:55:00,649 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,649 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106 2023-07-24 07:55:00,649 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a 2023-07-24 07:55:00,651 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/recovered.edits] 2023-07-24 07:55:00,652 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/recovered.edits] 2023-07-24 07:55:00,652 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/recovered.edits] 2023-07-24 07:55:00,655 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/.regioninfo to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/.regioninfo 2023-07-24 07:55:00,655 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/.regioninfo to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/.regioninfo 2023-07-24 07:55:00,656 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/02647ff08c7b4d408c45334940799d3e 2023-07-24 07:55:00,658 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/19b141de763a465397e1aa9b2273b8f3 2023-07-24 07:55:00,658 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,659 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/02647ff08c7b4d408c45334940799d3e.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,659 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/29e5c4e4f727419b83f9f455dbf87c84 2023-07-24 07:55:00,660 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,660 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/19b141de763a465397e1aa9b2273b8f3.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,661 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/3d44f2d8f3794b469861a10f9b1150d2 2023-07-24 07:55:00,661 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,661 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/29e5c4e4f727419b83f9f455dbf87c84.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,662 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/4226e054714c4683952791276791f341 2023-07-24 07:55:00,663 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,663 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/3d44f2d8f3794b469861a10f9b1150d2.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,664 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/99612eba074c4839ac421fe8b8d405d5 2023-07-24 07:55:00,664 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,665 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/4226e054714c4683952791276791f341.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,665 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf1/d7d33c4ba08947f8b752c158aa4b9c59 2023-07-24 07:55:00,666 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,666 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/99612eba074c4839ac421fe8b8d405d5.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,667 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,668 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf1/d7d33c4ba08947f8b752c158aa4b9c59.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,669 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/30b4d485121341688f91345ed26d6627 2023-07-24 07:55:00,671 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,671 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/93c6c7a1e37844e18310d5d23ed7fbfb 2023-07-24 07:55:00,672 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/30b4d485121341688f91345ed26d6627.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,672 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,673 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/a32fa3ab81384813881ecde578c4e4d0 2023-07-24 07:55:00,673 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/93c6c7a1e37844e18310d5d23ed7fbfb.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,674 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,674 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/ca7683188a754a688ee85e220c336277 2023-07-24 07:55:00,675 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/a32fa3ab81384813881ecde578c4e4d0.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,675 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,676 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d12defefae7341b5910a940e956e7f48 2023-07-24 07:55:00,676 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/ca7683188a754a688ee85e220c336277.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,676 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,677 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/d531a392fa1e475bac471f0201838a5f 2023-07-24 07:55:00,677 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/d12defefae7341b5910a940e956e7f48.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,677 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,678 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/cf2/f759ed834f994113bdfb5ee3f8566c56 2023-07-24 07:55:00,679 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/d531a392fa1e475bac471f0201838a5f.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,679 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/2123d27523d3bca109f37599c76acc9a/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,680 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/488ee2c34de56e3764da09c2c2def106/cf2/f759ed834f994113bdfb5ee3f8566c56.63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,681 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/7b9b3680e6c54abe97585e941f6d5624 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf1/7b9b3680e6c54abe97585e941f6d5624 2023-07-24 07:55:00,682 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/recovered.edits/85.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d/recovered.edits/85.seqid 2023-07-24 07:55:00,683 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/63edc6e611d2d72dc562948ea214a40d 2023-07-24 07:55:00,687 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/bc4a948fc4cc46c38ef007c434d7c046 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf1/bc4a948fc4cc46c38ef007c434d7c046 2023-07-24 07:55:00,688 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/b4cd007ae7fe458b969e7b28d9f3bd50 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/cf2/b4cd007ae7fe458b969e7b28d9f3bd50 2023-07-24 07:55:00,690 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/903d14b9d1e44e16be51786de47d5416 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/cf2/903d14b9d1e44e16be51786de47d5416 2023-07-24 07:55:00,691 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/recovered.edits/90.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a/recovered.edits/90.seqid 2023-07-24 07:55:00,691 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/2123d27523d3bca109f37599c76acc9a 2023-07-24 07:55:00,692 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/recovered.edits/90.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106/recovered.edits/90.seqid 2023-07-24 07:55:00,693 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testRecoveryAndDoubleExecution/488ee2c34de56e3764da09c2c2def106 2023-07-24 07:55:00,693 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-07-24 07:55:00,695 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:55:00,697 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-07-24 07:55:00,698 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-07-24 07:55:00,699 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:55:00,699 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-07-24 07:55:00,700 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185300699"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:00,700 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185300699"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:00,700 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185300699"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:00,701 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 07:55:00,701 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 63edc6e611d2d72dc562948ea214a40d, NAME => 'testRecoveryAndDoubleExecution,,1690185294381.63edc6e611d2d72dc562948ea214a40d.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 488ee2c34de56e3764da09c2c2def106, NAME => 'testRecoveryAndDoubleExecution,,1690185296239.488ee2c34de56e3764da09c2c2def106.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 2123d27523d3bca109f37599c76acc9a, NAME => 'testRecoveryAndDoubleExecution,41,1690185296239.2123d27523d3bca109f37599c76acc9a.', STARTKEY => '41', ENDKEY => ''}] 2023-07-24 07:55:00,701 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-07-24 07:55:00,701 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690185300701"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:00,703 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-07-24 07:55:00,704 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 07:55:00,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 62 msec 2023-07-24 07:55:00,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-07-24 07:55:00,746 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-07-24 07:55:00,763 INFO [Listener at localhost/42537] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=378 (was 371) - Thread LEAK? -, OpenFileDescriptor=835 (was 748) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=157 (was 162), ProcessCount=175 (was 171) - ProcessCount LEAK? -, AvailableMemoryMB=8839 (was 8887) 2023-07-24 07:55:00,779 INFO [Listener at localhost/42537] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=378, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=157, ProcessCount=175, AvailableMemoryMB=8838 2023-07-24 07:55:00,779 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:55:00,783 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 07:55:00,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 07:55:00,785 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 07:55:00,787 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 07:55:00,787 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 07:55:00,789 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:00,789 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4 empty. 2023-07-24 07:55:00,789 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:00,789 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-07-24 07:55:00,798 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-07-24 07:55:00,799 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 7cbead548a1582f81b9490192fcb37a4, NAME => 'testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp 2023-07-24 07:55:00,806 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:55:00,806 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing 7cbead548a1582f81b9490192fcb37a4, disabling compactions & flushes 2023-07-24 07:55:00,806 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:00,806 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:00,806 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. after waiting 0 ms 2023-07-24 07:55:00,806 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:00,806 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:00,806 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 7cbead548a1582f81b9490192fcb37a4: 2023-07-24 07:55:00,808 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 07:55:00,809 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185300809"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185300809"}]},"ts":"1690185300809"} 2023-07-24 07:55:00,810 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 07:55:00,811 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 07:55:00,811 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185300811"}]},"ts":"1690185300811"} 2023-07-24 07:55:00,812 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-07-24 07:55:00,814 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:55:00,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:55:00,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:55:00,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:55:00,815 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:55:00,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7cbead548a1582f81b9490192fcb37a4, ASSIGN}] 2023-07-24 07:55:00,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7cbead548a1582f81b9490192fcb37a4, ASSIGN 2023-07-24 07:55:00,817 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7cbead548a1582f81b9490192fcb37a4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46877,1690185199045; forceNewPlan=false, retain=false 2023-07-24 07:55:00,967 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:55:00,969 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=7cbead548a1582f81b9490192fcb37a4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:00,969 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185300969"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185300969"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185300969"}]},"ts":"1690185300969"} 2023-07-24 07:55:00,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 7cbead548a1582f81b9490192fcb37a4, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:55:01,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:01,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7cbead548a1582f81b9490192fcb37a4, NAME => 'testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:55:01,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:55:01,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:01,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:55:01,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:01,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:01,129 INFO [StoreOpener-7cbead548a1582f81b9490192fcb37a4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:01,130 DEBUG [StoreOpener-7cbead548a1582f81b9490192fcb37a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1 2023-07-24 07:55:01,130 DEBUG [StoreOpener-7cbead548a1582f81b9490192fcb37a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1 2023-07-24 07:55:01,131 INFO [StoreOpener-7cbead548a1582f81b9490192fcb37a4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7cbead548a1582f81b9490192fcb37a4 columnFamilyName cf1 2023-07-24 07:55:01,131 INFO [StoreOpener-7cbead548a1582f81b9490192fcb37a4-1] regionserver.HStore(310): Store=7cbead548a1582f81b9490192fcb37a4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:01,131 INFO [StoreOpener-7cbead548a1582f81b9490192fcb37a4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:01,132 DEBUG [StoreOpener-7cbead548a1582f81b9490192fcb37a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2 2023-07-24 07:55:01,132 DEBUG [StoreOpener-7cbead548a1582f81b9490192fcb37a4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2 2023-07-24 07:55:01,133 INFO [StoreOpener-7cbead548a1582f81b9490192fcb37a4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7cbead548a1582f81b9490192fcb37a4 columnFamilyName cf2 2023-07-24 07:55:01,133 INFO [StoreOpener-7cbead548a1582f81b9490192fcb37a4-1] regionserver.HStore(310): Store=7cbead548a1582f81b9490192fcb37a4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:01,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:01,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:01,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:55:01,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:01,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:55:01,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7cbead548a1582f81b9490192fcb37a4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75363345, jitterRate=0.1230013519525528}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:55:01,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7cbead548a1582f81b9490192fcb37a4: 2023-07-24 07:55:01,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4., pid=127, masterSystemTime=1690185301122 2023-07-24 07:55:01,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:01,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:01,140 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=7cbead548a1582f81b9490192fcb37a4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,140 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185301140"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185301140"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185301140"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185301140"}]},"ts":"1690185301140"} 2023-07-24 07:55:01,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-07-24 07:55:01,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 7cbead548a1582f81b9490192fcb37a4, server=jenkins-hbase4.apache.org,46877,1690185199045 in 171 msec 2023-07-24 07:55:01,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-07-24 07:55:01,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7cbead548a1582f81b9490192fcb37a4, ASSIGN in 328 msec 2023-07-24 07:55:01,144 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 07:55:01,145 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185301144"}]},"ts":"1690185301144"} 2023-07-24 07:55:01,145 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-07-24 07:55:01,147 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 07:55:01,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 363 msec 2023-07-24 07:55:01,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:01,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,294 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:01,294 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,294 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 07:55:01,294 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 07:55:01,294 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,294 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,297 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,297 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,297 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,297 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,297 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,297 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:01,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,298 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,298 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,298 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,298 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:01,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:01,299 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,299 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,299 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,299 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:01,299 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,299 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. started... 2023-07-24 07:55:01,299 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:01,300 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:01,301 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7cbead548a1582f81b9490192fcb37a4 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 07:55:01,301 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,301 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,301 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:01,302 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,302 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,302 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:01,310 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/cfbbd450c7794f6fa3128ac2c9c765ea 2023-07-24 07:55:01,321 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/6479114b8e44449c9ea70d6f169bffbb 2023-07-24 07:55:01,325 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/cfbbd450c7794f6fa3128ac2c9c765ea as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cfbbd450c7794f6fa3128ac2c9c765ea 2023-07-24 07:55:01,329 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cfbbd450c7794f6fa3128ac2c9c765ea, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:55:01,329 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/6479114b8e44449c9ea70d6f169bffbb as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/6479114b8e44449c9ea70d6f169bffbb 2023-07-24 07:55:01,333 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/6479114b8e44449c9ea70d6f169bffbb, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:55:01,334 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7cbead548a1582f81b9490192fcb37a4 in 33ms, sequenceid=6, compaction requested=false 2023-07-24 07:55:01,334 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:01,335 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7cbead548a1582f81b9490192fcb37a4: 2023-07-24 07:55:01,335 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:01,335 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:01,335 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:01,335 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,335 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:01,335 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:01,337 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,337 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:01,337 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,337 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:01,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:01,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 07:55:01,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1be7a21b[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:01,339 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 07:55:01,339 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,341 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,341 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,341 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:01,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,341 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:01,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:01,341 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:01,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:01,341 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,343 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:01,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:01,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:55:01,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:55:01,346 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 07:55:01,346 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:01,346 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:01,346 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,346 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:01,347 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,347 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,347 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,347 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:01,347 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,347 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,348 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,348 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:01,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,348 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,349 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:01,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:01,350 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,351 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,351 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,359 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,359 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,359 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,359 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,359 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,359 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,359 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,359 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:01,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:01,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:01,360 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,461 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:01,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:01,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:01,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:55:01,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,474 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:01,474 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,474 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 07:55:01,474 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 07:55:01,474 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,474 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,475 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,476 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,476 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,475 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,476 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,476 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:01,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,476 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:01,477 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,477 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,477 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,477 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:01,478 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:01,478 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:01,478 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. started... 2023-07-24 07:55:01,478 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7cbead548a1582f81b9490192fcb37a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:55:01,479 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,479 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,479 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,479 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,480 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:01,480 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:01,487 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/cc695c8e1d704c40ba02dde83270ee37 2023-07-24 07:55:01,499 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/645a11ffca154d47b5c8dd43599d74b5 2023-07-24 07:55:01,503 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/cc695c8e1d704c40ba02dde83270ee37 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cc695c8e1d704c40ba02dde83270ee37 2023-07-24 07:55:01,507 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cc695c8e1d704c40ba02dde83270ee37, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:55:01,508 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/645a11ffca154d47b5c8dd43599d74b5 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/645a11ffca154d47b5c8dd43599d74b5 2023-07-24 07:55:01,511 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/645a11ffca154d47b5c8dd43599d74b5, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:55:01,512 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7cbead548a1582f81b9490192fcb37a4 in 34ms, sequenceid=19, compaction requested=false 2023-07-24 07:55:01,512 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7cbead548a1582f81b9490192fcb37a4: 2023-07-24 07:55:01,512 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:01,512 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:01,512 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:01,512 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,512 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:01,512 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:01,514 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,514 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:01,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:01,514 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,514 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:01,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:01,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:01,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:01,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 07:55:01,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@688c58a[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:01,516 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 07:55:01,516 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,518 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,518 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,518 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,518 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:01,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:01,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:01,519 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:01,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:01,519 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:01,520 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:01,520 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,520 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:01,520 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,520 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:01,520 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:01,521 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:01,521 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,521 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,521 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:01,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:01,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:01,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:01,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:01,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:01,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:55:01,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:55:01,524 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 07:55:01,524 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:01,524 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:01,524 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,524 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:01,525 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,525 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:01,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:01,525 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,525 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,525 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,526 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:01,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,526 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,526 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,527 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:01,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,527 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:01,528 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,529 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,529 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,532 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,532 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,532 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,533 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,533 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,532 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,532 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,533 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,533 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:01,533 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:01,533 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,533 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,533 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,533 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,533 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:01,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,533 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,533 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:01,534 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,534 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,534 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,534 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,610 ERROR [BP-80517894-172.31.14.131-1690185194253 heartbeating to localhost/127.0.0.1:34211] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-80517894-172.31.14.131-1690185194253 (Datanode Uuid 1d2b2de7-56a0-45bb-9a79-d5935e59fefd) service to localhost/127.0.0.1:34211 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-24 07:55:01,634 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:01,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:01,646 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:01,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:55:01,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,647 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:01,648 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,648 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 07:55:01,648 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 07:55:01,648 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,648 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,649 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,649 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,649 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,649 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,650 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,650 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:01,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,650 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,651 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:01,651 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,651 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,651 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,651 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:01,652 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:01,652 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:01,652 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,652 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:01,652 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:01,652 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:01,652 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. started... 2023-07-24 07:55:01,653 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7cbead548a1582f81b9490192fcb37a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:55:01,653 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,653 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,653 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:01,654 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,654 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,655 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:01,663 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/23399b5b9cff420e8895f747f5ec5459 2023-07-24 07:55:01,674 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/93d29de62f4347929332b3edf808ce09 2023-07-24 07:55:01,677 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/23399b5b9cff420e8895f747f5ec5459 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/23399b5b9cff420e8895f747f5ec5459 2023-07-24 07:55:01,680 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/23399b5b9cff420e8895f747f5ec5459, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:55:01,681 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/93d29de62f4347929332b3edf808ce09 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/93d29de62f4347929332b3edf808ce09 2023-07-24 07:55:01,684 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/93d29de62f4347929332b3edf808ce09, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:55:01,685 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7cbead548a1582f81b9490192fcb37a4 in 33ms, sequenceid=32, compaction requested=true 2023-07-24 07:55:01,685 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7cbead548a1582f81b9490192fcb37a4: 2023-07-24 07:55:01,685 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:01,686 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:01,686 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:01,686 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,686 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:01,686 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:01,692 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:01,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:01,692 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,692 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:01,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:01,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:01,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:01,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 07:55:01,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6446d46a[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:01,694 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 07:55:01,694 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,695 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,695 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,695 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,695 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:01,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,695 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:01,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,695 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,695 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:01,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,695 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:01,695 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:01,696 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:01,696 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:01,695 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:01,695 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:01,696 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:01,699 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:01,699 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,699 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:01,699 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:01,699 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,699 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,699 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:01,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,699 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,699 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:01,700 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:01,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:01,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:01,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:01,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:01,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:01,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:55:01,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:55:01,703 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 07:55:01,703 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:01,703 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:01,703 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,703 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:01,705 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,705 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,705 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,705 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,705 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,705 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,705 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:01,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:01,706 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:01,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,706 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,707 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:01,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,708 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:01,708 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,709 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,711 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,711 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,711 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,711 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,711 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,711 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:01,711 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,711 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:01,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:01,711 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,712 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,712 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:01,712 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,712 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,712 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:01,712 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,712 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:01,712 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:01,712 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,713 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,713 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:01,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:01,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:01,812 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:01,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:01,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:01,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:55:01,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,825 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:01,825 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,826 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 07:55:01,826 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 07:55:01,826 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,826 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,827 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,827 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,827 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,827 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:01,827 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:01,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:01,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:01,827 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,828 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,828 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,828 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,828 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:01,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,829 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:01,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,829 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,829 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,829 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,829 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:01,829 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,829 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,829 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:01,829 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,829 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. started... 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:01,829 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7cbead548a1582f81b9490192fcb37a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:01,829 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:01,830 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:01,831 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,831 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,832 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,832 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:01,832 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:01,832 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:02,239 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/9587d916339845b48d6533ad1cc97b18 2023-07-24 07:55:02,650 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/5bb1c3306ce74278878ca12a3acb8af0 2023-07-24 07:55:02,655 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/9587d916339845b48d6533ad1cc97b18 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/9587d916339845b48d6533ad1cc97b18 2023-07-24 07:55:02,658 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/9587d916339845b48d6533ad1cc97b18, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:55:02,659 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/5bb1c3306ce74278878ca12a3acb8af0 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/5bb1c3306ce74278878ca12a3acb8af0 2023-07-24 07:55:02,662 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/5bb1c3306ce74278878ca12a3acb8af0, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:55:02,663 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7cbead548a1582f81b9490192fcb37a4 in 834ms, sequenceid=45, compaction requested=true 2023-07-24 07:55:02,663 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7cbead548a1582f81b9490192fcb37a4: 2023-07-24 07:55:02,663 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:02,663 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:02,664 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,664 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:02,664 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:02,666 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,666 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:02,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:02,666 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,666 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:02,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:02,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:02,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:02,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:02,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:02,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 07:55:02,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@112cb9c[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:02,668 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 07:55:02,668 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,669 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,669 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,669 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:02,670 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,670 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:02,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:02,670 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:02,671 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:02,671 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:02,671 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,671 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:02,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:02,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:02,672 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:02,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:02,672 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:02,672 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:02,672 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:02,672 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:02,672 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:02,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:02,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:02,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:02,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:02,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:02,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:02,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:55:02,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:55:02,676 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 07:55:02,677 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:02,677 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:02,677 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,677 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:02,679 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,679 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:02,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:02,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,679 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:02,679 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,679 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,679 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:02,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:02,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,679 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:02,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:02,680 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:02,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:02,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:02,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:02,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:02,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:02,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,680 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:02,681 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:02,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:02,682 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,683 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:02,683 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:02,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:02,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:02,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:02,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:02,692 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:02,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:02,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:02,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:02,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:02,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:02,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:02,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:02,692 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:02,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:02,693 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:02,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:02,692 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:02,693 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:02,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:02,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:02,693 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:02,693 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,693 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,693 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:02,693 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,693 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,739 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 07:55:02,793 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:02,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:02,804 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:02,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:55:02,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,806 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:02,806 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:02,806 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 07:55:02,806 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 07:55:02,806 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,806 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,808 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:02,808 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:02,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:02,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:02,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:02,808 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,808 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:02,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:02,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:02,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:02,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,808 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:02,808 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:02,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,809 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:02,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,809 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,809 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:02,809 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,809 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:02,809 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:02,809 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:02,809 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:02,809 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:02,810 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:02,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,810 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,811 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:02,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,811 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,811 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,811 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:02,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:02,812 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,812 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:02,812 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:02,812 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:02,812 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:02,812 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. started... 2023-07-24 07:55:02,813 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7cbead548a1582f81b9490192fcb37a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:55:02,814 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,814 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:02,814 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:03,222 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/be9823364d854b7a929cc9908d81d349 2023-07-24 07:55:03,613 ERROR [BP-80517894-172.31.14.131-1690185194253 heartbeating to localhost/127.0.0.1:34211] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-80517894-172.31.14.131-1690185194253 (Datanode Uuid cda8ebfd-f055-417e-8edc-61fd19134adf) service to localhost/127.0.0.1:34211 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-24 07:55:03,637 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/0d6c9e60733846d79d00324bb180262b 2023-07-24 07:55:03,642 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/be9823364d854b7a929cc9908d81d349 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/be9823364d854b7a929cc9908d81d349 2023-07-24 07:55:03,645 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/be9823364d854b7a929cc9908d81d349, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:55:03,646 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/0d6c9e60733846d79d00324bb180262b as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/0d6c9e60733846d79d00324bb180262b 2023-07-24 07:55:03,649 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/0d6c9e60733846d79d00324bb180262b, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:55:03,650 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7cbead548a1582f81b9490192fcb37a4 in 837ms, sequenceid=58, compaction requested=true 2023-07-24 07:55:03,650 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7cbead548a1582f81b9490192fcb37a4: 2023-07-24 07:55:03,650 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:03,650 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:03,650 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:03,650 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,650 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:03,650 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:03,653 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,653 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:03,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:03,653 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,653 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:03,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:03,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:03,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:03,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:03,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:03,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 07:55:03,655 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27c356df[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:03,655 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 07:55:03,655 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,656 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,656 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,656 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,656 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:03,657 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:03,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,657 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:03,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,657 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:03,657 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:03,657 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:03,657 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,657 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:03,657 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:03,657 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:03,657 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:03,658 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:03,658 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,658 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:03,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,658 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:03,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:03,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:03,658 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:03,658 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:03,659 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:03,659 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:03,659 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:03,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:03,659 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:03,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:03,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:03,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:03,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:03,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:03,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:03,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:55:03,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:55:03,662 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 07:55:03,662 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:03,662 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:03,662 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,662 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:03,665 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,665 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,665 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,665 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:03,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:03,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:03,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,665 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,665 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,665 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:03,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,666 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:03,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:03,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:03,666 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:03,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:03,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:03,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:03,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:03,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:03,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,666 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:03,667 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:03,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:03,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:03,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:03,668 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:03,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,668 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:03,669 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:03,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:03,671 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:03,671 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:03,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:03,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:03,671 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:03,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:03,671 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:03,671 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:03,671 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:03,671 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:03,671 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:03,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:03,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:03,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:03,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:03,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:03,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,672 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:03,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:03,772 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:03,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:03,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:03,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 07:55:03,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,785 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:03,785 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:03,785 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 07:55:03,785 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 07:55:03,786 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,786 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,787 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:03,787 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:03,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:03,787 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:03,787 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:03,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:03,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:03,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,788 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:03,788 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:03,788 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,788 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,788 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,789 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,789 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,789 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,789 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,789 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:03,789 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 07:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,789 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 07:55:03,789 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:03,790 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:03,790 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:03,790 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:03,790 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:03,790 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. started... 2023-07-24 07:55:03,790 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7cbead548a1582f81b9490192fcb37a4 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:55:03,791 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,791 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,792 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,792 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:03,792 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:03,792 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:04,201 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/5e8e995648c8441da272db6122f183e9 2023-07-24 07:55:04,213 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/b14448b285f047ed8f6c4596dccb7675 2023-07-24 07:55:04,216 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/5e8e995648c8441da272db6122f183e9 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/5e8e995648c8441da272db6122f183e9 2023-07-24 07:55:04,220 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/5e8e995648c8441da272db6122f183e9, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:55:04,220 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/b14448b285f047ed8f6c4596dccb7675 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/b14448b285f047ed8f6c4596dccb7675 2023-07-24 07:55:04,224 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/b14448b285f047ed8f6c4596dccb7675, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:55:04,224 INFO [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7cbead548a1582f81b9490192fcb37a4 in 434ms, sequenceid=71, compaction requested=true 2023-07-24 07:55:04,224 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7cbead548a1582f81b9490192fcb37a4: 2023-07-24 07:55:04,225 DEBUG [rs(jenkins-hbase4.apache.org,46877,1690185199045)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:04,225 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:04,225 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:04,225 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,225 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 07:55:04,225 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 07:55:04,227 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,227 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:04,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:04,227 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,228 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:04,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:04,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:04,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:04,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:04,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:04,229 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 07:55:04,229 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67277338[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:04,229 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 07:55:04,229 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,233 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,233 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,233 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,233 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,233 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:04,233 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:04,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,233 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:04,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,233 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:04,234 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:04,234 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:04,234 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:04,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,234 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 07:55:04,234 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 07:55:04,234 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:04,235 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:04,235 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:04,235 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:04,237 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,237 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:04,237 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:04,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,237 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:04,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:04,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:04,238 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 07:55:04,238 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:04,238 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 07:55:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:04,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:04,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:04,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:04,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:04,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:04,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,46877,1690185199045': 2023-07-24 07:55:04,241 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 07:55:04,241 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:04,241 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 07:55:04,241 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:04,241 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,241 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:04,242 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,242 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,242 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:04,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:04,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,242 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:04,242 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:04,242 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:04,243 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:04,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:04,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:04,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,243 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:04,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:04,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:04,243 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:04,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:04,243 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:04,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:04,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:04,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:04,245 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:04,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:04,245 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,245 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:04,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:04,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:04,248 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:04,248 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:04,248 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,248 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:04,248 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:04,248 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:04,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:04,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:04,248 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:04,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 07:55:04,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:04,248 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:04,248 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:04,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:04,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:04,248 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:04,249 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:04,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:04,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:04,248 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:04,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:04,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:04,249 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:04,249 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,249 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:04,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:04,249 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,249 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,249 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:04,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:04,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:04,249 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,249 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 07:55:04,349 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:04,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:04,358 DEBUG [Listener at localhost/42537] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,359 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=7cbead548a1582f81b9490192fcb37a4, daughterA=6159e36da8f6c73d8177848cbd8e3fce, daughterB=44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:04,360 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=7cbead548a1582f81b9490192fcb37a4, daughterA=6159e36da8f6c73d8177848cbd8e3fce, daughterB=44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:04,360 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=7cbead548a1582f81b9490192fcb37a4, daughterA=6159e36da8f6c73d8177848cbd8e3fce, daughterB=44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:04,360 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=7cbead548a1582f81b9490192fcb37a4, daughterA=6159e36da8f6c73d8177848cbd8e3fce, daughterB=44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:04,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7cbead548a1582f81b9490192fcb37a4, UNASSIGN}] 2023-07-24 07:55:04,362 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7cbead548a1582f81b9490192fcb37a4, UNASSIGN 2023-07-24 07:55:04,362 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=7cbead548a1582f81b9490192fcb37a4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:04,363 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185304362"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185304362"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185304362"}]},"ts":"1690185304362"} 2023-07-24 07:55:04,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 7cbead548a1582f81b9490192fcb37a4, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:55:04,515 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,515 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7cbead548a1582f81b9490192fcb37a4, disabling compactions & flushes 2023-07-24 07:55:04,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:04,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:04,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. after waiting 0 ms 2023-07-24 07:55:04,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:04,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 7cbead548a1582f81b9490192fcb37a4 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 07:55:04,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/e4f68f702ae94e14b25162e48ca9ea7f 2023-07-24 07:55:04,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/18b9b7dc95134b0da428d14465b6d07d 2023-07-24 07:55:04,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf1/e4f68f702ae94e14b25162e48ca9ea7f as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/e4f68f702ae94e14b25162e48ca9ea7f 2023-07-24 07:55:04,541 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/e4f68f702ae94e14b25162e48ca9ea7f, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:55:04,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.tmp/cf2/18b9b7dc95134b0da428d14465b6d07d as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/18b9b7dc95134b0da428d14465b6d07d 2023-07-24 07:55:04,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/18b9b7dc95134b0da428d14465b6d07d, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:55:04,546 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7cbead548a1582f81b9490192fcb37a4 in 30ms, sequenceid=82, compaction requested=true 2023-07-24 07:55:04,550 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 07:55:04,550 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:55:04,551 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. 2023-07-24 07:55:04,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7cbead548a1582f81b9490192fcb37a4: 2023-07-24 07:55:04,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,553 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=7cbead548a1582f81b9490192fcb37a4, regionState=CLOSED 2023-07-24 07:55:04,553 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185304553"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185304553"}]},"ts":"1690185304553"} 2023-07-24 07:55:04,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-07-24 07:55:04,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 7cbead548a1582f81b9490192fcb37a4, server=jenkins-hbase4.apache.org,46877,1690185199045 in 190 msec 2023-07-24 07:55:04,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-07-24 07:55:04,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=7cbead548a1582f81b9490192fcb37a4, UNASSIGN in 195 msec 2023-07-24 07:55:04,567 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=7cbead548a1582f81b9490192fcb37a4, threads=14 2023-07-24 07:55:04,568 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/0d6c9e60733846d79d00324bb180262b for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,568 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/18b9b7dc95134b0da428d14465b6d07d for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,568 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/5bb1c3306ce74278878ca12a3acb8af0 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,568 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/645a11ffca154d47b5c8dd43599d74b5 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,568 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/6479114b8e44449c9ea70d6f169bffbb for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,570 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/93d29de62f4347929332b3edf808ce09 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,571 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/b14448b285f047ed8f6c4596dccb7675 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,571 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/23399b5b9cff420e8895f747f5ec5459 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,571 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/5e8e995648c8441da272db6122f183e9 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,571 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/9587d916339845b48d6533ad1cc97b18 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,571 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/be9823364d854b7a929cc9908d81d349 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,572 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cc695c8e1d704c40ba02dde83270ee37 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,574 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cfbbd450c7794f6fa3128ac2c9c765ea for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,575 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/e4f68f702ae94e14b25162e48ca9ea7f for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,578 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/5bb1c3306ce74278878ca12a3acb8af0, top=false 2023-07-24 07:55:04,587 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/6479114b8e44449c9ea70d6f169bffbb, top=false 2023-07-24 07:55:04,587 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/645a11ffca154d47b5c8dd43599d74b5, top=false 2023-07-24 07:55:04,588 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/0d6c9e60733846d79d00324bb180262b, top=false 2023-07-24 07:55:04,588 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/18b9b7dc95134b0da428d14465b6d07d, top=false 2023-07-24 07:55:04,589 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/9587d916339845b48d6533ad1cc97b18, top=false 2023-07-24 07:55:04,589 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/b14448b285f047ed8f6c4596dccb7675, top=false 2023-07-24 07:55:04,589 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/5e8e995648c8441da272db6122f183e9, top=false 2023-07-24 07:55:04,591 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/e4f68f702ae94e14b25162e48ca9ea7f, top=false 2023-07-24 07:55:04,592 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/93d29de62f4347929332b3edf808ce09, top=false 2023-07-24 07:55:04,593 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/be9823364d854b7a929cc9908d81d349, top=false 2023-07-24 07:55:04,593 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cc695c8e1d704c40ba02dde83270ee37, top=false 2023-07-24 07:55:04,593 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cfbbd450c7794f6fa3128ac2c9c765ea, top=false 2023-07-24 07:55:04,594 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/23399b5b9cff420e8895f747f5ec5459, top=false 2023-07-24 07:55:04,615 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-18b9b7dc95134b0da428d14465b6d07d for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,619 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5e8e995648c8441da272db6122f183e9 for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,619 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5bb1c3306ce74278878ca12a3acb8af0 for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,619 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/18b9b7dc95134b0da428d14465b6d07d for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,620 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-0d6c9e60733846d79d00324bb180262b for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,620 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-be9823364d854b7a929cc9908d81d349 for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,620 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cc695c8e1d704c40ba02dde83270ee37 for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,620 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-6479114b8e44449c9ea70d6f169bffbb for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,620 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-e4f68f702ae94e14b25162e48ca9ea7f for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,623 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/5e8e995648c8441da272db6122f183e9 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,625 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/5bb1c3306ce74278878ca12a3acb8af0 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,625 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cc695c8e1d704c40ba02dde83270ee37 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,625 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/be9823364d854b7a929cc9908d81d349 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,626 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/e4f68f702ae94e14b25162e48ca9ea7f for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,627 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/0d6c9e60733846d79d00324bb180262b for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,628 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/6479114b8e44449c9ea70d6f169bffbb for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,632 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-93d29de62f4347929332b3edf808ce09 for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,632 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-645a11ffca154d47b5c8dd43599d74b5 for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,633 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-9587d916339845b48d6533ad1cc97b18 for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,635 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cfbbd450c7794f6fa3128ac2c9c765ea for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,635 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-b14448b285f047ed8f6c4596dccb7675 for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,636 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/93d29de62f4347929332b3edf808ce09 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,637 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/9587d916339845b48d6533ad1cc97b18 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,638 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/.splits/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-23399b5b9cff420e8895f747f5ec5459 for child: 6159e36da8f6c73d8177848cbd8e3fce, parent: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,638 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/645a11ffca154d47b5c8dd43599d74b5 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,639 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cfbbd450c7794f6fa3128ac2c9c765ea for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,639 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/b14448b285f047ed8f6c4596dccb7675 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,641 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/23399b5b9cff420e8895f747f5ec5459 for region: 7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:04,641 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 7cbead548a1582f81b9490192fcb37a4 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-07-24 07:55:05,072 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:55:05,074 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:55:05,076 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690185305076"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1690185305076"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1690185305076"}]},"ts":"1690185305076"} 2023-07-24 07:55:05,076 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690185305076"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185305076"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185305076"}]},"ts":"1690185305076"} 2023-07-24 07:55:05,076 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690185305076"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185305076"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185305076"}]},"ts":"1690185305076"} 2023-07-24 07:55:05,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6159e36da8f6c73d8177848cbd8e3fce, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=44855eaf1c1cbb0be3a6c7e3ed8bb286, ASSIGN}] 2023-07-24 07:55:05,080 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=44855eaf1c1cbb0be3a6c7e3ed8bb286, ASSIGN 2023-07-24 07:55:05,080 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=44855eaf1c1cbb0be3a6c7e3ed8bb286, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46877,1690185199045; forceNewPlan=false, retain=false 2023-07-24 07:55:05,081 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6159e36da8f6c73d8177848cbd8e3fce, ASSIGN 2023-07-24 07:55:05,081 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6159e36da8f6c73d8177848cbd8e3fce, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46877,1690185199045; forceNewPlan=false, retain=false 2023-07-24 07:55:05,230 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 07:55:05,232 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=44855eaf1c1cbb0be3a6c7e3ed8bb286, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:05,232 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690185305232"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185305232"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185305232"}]},"ts":"1690185305232"} 2023-07-24 07:55:05,233 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 44855eaf1c1cbb0be3a6c7e3ed8bb286, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:55:05,236 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=6159e36da8f6c73d8177848cbd8e3fce, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:05,236 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690185305236"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185305236"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185305236"}]},"ts":"1690185305236"} 2023-07-24 07:55:05,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 6159e36da8f6c73d8177848cbd8e3fce, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:55:05,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286. 2023-07-24 07:55:05,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 44855eaf1c1cbb0be3a6c7e3ed8bb286, NAME => 'testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286.', STARTKEY => '71', ENDKEY => ''} 2023-07-24 07:55:05,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:55:05,388 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:05,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:55:05,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:05,389 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:05,390 INFO [StoreOpener-44855eaf1c1cbb0be3a6c7e3ed8bb286-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:05,391 DEBUG [StoreOpener-44855eaf1c1cbb0be3a6c7e3ed8bb286-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286/cf1 2023-07-24 07:55:05,391 DEBUG [StoreOpener-44855eaf1c1cbb0be3a6c7e3ed8bb286-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286/cf1 2023-07-24 07:55:05,392 INFO [StoreOpener-44855eaf1c1cbb0be3a6c7e3ed8bb286-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 44855eaf1c1cbb0be3a6c7e3ed8bb286 columnFamilyName cf1 2023-07-24 07:55:05,392 INFO [StoreOpener-44855eaf1c1cbb0be3a6c7e3ed8bb286-1] regionserver.HStore(310): Store=44855eaf1c1cbb0be3a6c7e3ed8bb286/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:05,392 INFO [StoreOpener-44855eaf1c1cbb0be3a6c7e3ed8bb286-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:05,393 DEBUG [StoreOpener-44855eaf1c1cbb0be3a6c7e3ed8bb286-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286/cf2 2023-07-24 07:55:05,393 DEBUG [StoreOpener-44855eaf1c1cbb0be3a6c7e3ed8bb286-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286/cf2 2023-07-24 07:55:05,394 INFO [StoreOpener-44855eaf1c1cbb0be3a6c7e3ed8bb286-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 44855eaf1c1cbb0be3a6c7e3ed8bb286 columnFamilyName cf2 2023-07-24 07:55:05,394 INFO [StoreOpener-44855eaf1c1cbb0be3a6c7e3ed8bb286-1] regionserver.HStore(310): Store=44855eaf1c1cbb0be3a6c7e3ed8bb286/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:05,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:05,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:05,398 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:55:05,399 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:05,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 44855eaf1c1cbb0be3a6c7e3ed8bb286; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73599287, jitterRate=0.09671483933925629}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:55:05,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 44855eaf1c1cbb0be3a6c7e3ed8bb286: 2023-07-24 07:55:05,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286., pid=133, masterSystemTime=1690185305385 2023-07-24 07:55:05,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286. 2023-07-24 07:55:05,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286. 2023-07-24 07:55:05,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce. 2023-07-24 07:55:05,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6159e36da8f6c73d8177848cbd8e3fce, NAME => 'testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.', STARTKEY => '', ENDKEY => '71'} 2023-07-24 07:55:05,402 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=44855eaf1c1cbb0be3a6c7e3ed8bb286, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:05,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:55:05,402 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690185305402"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185305402"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185305402"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185305402"}]},"ts":"1690185305402"} 2023-07-24 07:55:05,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:05,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:55:05,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:05,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:05,404 INFO [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:05,405 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1 2023-07-24 07:55:05,405 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1 2023-07-24 07:55:05,405 INFO [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6159e36da8f6c73d8177848cbd8e3fce columnFamilyName cf1 2023-07-24 07:55:05,411 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-23399b5b9cff420e8895f747f5ec5459 2023-07-24 07:55:05,414 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5e8e995648c8441da272db6122f183e9 2023-07-24 07:55:05,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-07-24 07:55:05,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 44855eaf1c1cbb0be3a6c7e3ed8bb286, server=jenkins-hbase4.apache.org,46877,1690185199045 in 171 msec 2023-07-24 07:55:05,417 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-9587d916339845b48d6533ad1cc97b18 2023-07-24 07:55:05,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=44855eaf1c1cbb0be3a6c7e3ed8bb286, ASSIGN in 337 msec 2023-07-24 07:55:05,420 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-be9823364d854b7a929cc9908d81d349 2023-07-24 07:55:05,423 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cc695c8e1d704c40ba02dde83270ee37 2023-07-24 07:55:05,425 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cfbbd450c7794f6fa3128ac2c9c765ea 2023-07-24 07:55:05,428 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-e4f68f702ae94e14b25162e48ca9ea7f 2023-07-24 07:55:05,428 INFO [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(310): Store=6159e36da8f6c73d8177848cbd8e3fce/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:05,428 INFO [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:05,430 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2 2023-07-24 07:55:05,430 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2 2023-07-24 07:55:05,430 INFO [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6159e36da8f6c73d8177848cbd8e3fce columnFamilyName cf2 2023-07-24 07:55:05,435 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-0d6c9e60733846d79d00324bb180262b 2023-07-24 07:55:05,437 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-18b9b7dc95134b0da428d14465b6d07d 2023-07-24 07:55:05,440 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5bb1c3306ce74278878ca12a3acb8af0 2023-07-24 07:55:05,442 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-645a11ffca154d47b5c8dd43599d74b5 2023-07-24 07:55:05,445 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-6479114b8e44449c9ea70d6f169bffbb 2023-07-24 07:55:05,447 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-93d29de62f4347929332b3edf808ce09 2023-07-24 07:55:05,450 DEBUG [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-b14448b285f047ed8f6c4596dccb7675 2023-07-24 07:55:05,450 INFO [StoreOpener-6159e36da8f6c73d8177848cbd8e3fce-1] regionserver.HStore(310): Store=6159e36da8f6c73d8177848cbd8e3fce/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:05,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:05,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:05,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:55:05,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:05,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6159e36da8f6c73d8177848cbd8e3fce; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67033645, jitterRate=-0.0011208504438400269}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:55:05,454 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6159e36da8f6c73d8177848cbd8e3fce: 2023-07-24 07:55:05,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce., pid=134, masterSystemTime=1690185305385 2023-07-24 07:55:05,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:05,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 07:55:05,456 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:55:05,456 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:55:05,456 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-24 07:55:05,456 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-24 07:55:05,456 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HStore(1912): 6159e36da8f6c73d8177848cbd8e3fce/cf2 is initiating minor compaction (all files) 2023-07-24 07:55:05,456 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HStore(1912): 6159e36da8f6c73d8177848cbd8e3fce/cf1 is initiating minor compaction (all files) 2023-07-24 07:55:05,456 INFO [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6159e36da8f6c73d8177848cbd8e3fce/cf2 in testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce. 2023-07-24 07:55:05,456 INFO [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6159e36da8f6c73d8177848cbd8e3fce/cf1 in testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce. 2023-07-24 07:55:05,457 INFO [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-6479114b8e44449c9ea70d6f169bffbb, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-645a11ffca154d47b5c8dd43599d74b5, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-93d29de62f4347929332b3edf808ce09, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5bb1c3306ce74278878ca12a3acb8af0, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-0d6c9e60733846d79d00324bb180262b, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-b14448b285f047ed8f6c4596dccb7675, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-18b9b7dc95134b0da428d14465b6d07d] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/.tmp, totalSize=34.9 K 2023-07-24 07:55:05,457 INFO [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cfbbd450c7794f6fa3128ac2c9c765ea, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cc695c8e1d704c40ba02dde83270ee37, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-23399b5b9cff420e8895f747f5ec5459, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-9587d916339845b48d6533ad1cc97b18, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-be9823364d854b7a929cc9908d81d349, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5e8e995648c8441da272db6122f183e9, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-e4f68f702ae94e14b25162e48ca9ea7f] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/.tmp, totalSize=34.9 K 2023-07-24 07:55:05,457 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-6479114b8e44449c9ea70d6f169bffbb, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185301288 2023-07-24 07:55:05,457 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cfbbd450c7794f6fa3128ac2c9c765ea, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185301288 2023-07-24 07:55:05,457 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-645a11ffca154d47b5c8dd43599d74b5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185301462 2023-07-24 07:55:05,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce. 2023-07-24 07:55:05,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce. 2023-07-24 07:55:05,458 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cc695c8e1d704c40ba02dde83270ee37, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185301462 2023-07-24 07:55:05,458 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-93d29de62f4347929332b3edf808ce09, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185301635 2023-07-24 07:55:05,458 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=6159e36da8f6c73d8177848cbd8e3fce, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:05,458 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690185305458"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185305458"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185305458"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185305458"}]},"ts":"1690185305458"} 2023-07-24 07:55:05,458 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-23399b5b9cff420e8895f747f5ec5459, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185301635 2023-07-24 07:55:05,458 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5bb1c3306ce74278878ca12a3acb8af0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185301814 2023-07-24 07:55:05,459 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-0d6c9e60733846d79d00324bb180262b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185302794 2023-07-24 07:55:05,459 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-9587d916339845b48d6533ad1cc97b18, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185301814 2023-07-24 07:55:05,459 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-b14448b285f047ed8f6c4596dccb7675, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185303774 2023-07-24 07:55:05,459 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-be9823364d854b7a929cc9908d81d349, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185302794 2023-07-24 07:55:05,459 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-18b9b7dc95134b0da428d14465b6d07d, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185304351 2023-07-24 07:55:05,460 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5e8e995648c8441da272db6122f183e9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185303774 2023-07-24 07:55:05,460 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-e4f68f702ae94e14b25162e48ca9ea7f, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185304351 2023-07-24 07:55:05,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-07-24 07:55:05,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 6159e36da8f6c73d8177848cbd8e3fce, server=jenkins-hbase4.apache.org,46877,1690185199045 in 222 msec 2023-07-24 07:55:05,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-07-24 07:55:05,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6159e36da8f6c73d8177848cbd8e3fce, ASSIGN in 382 msec 2023-07-24 07:55:05,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=7cbead548a1582f81b9490192fcb37a4, daughterA=6159e36da8f6c73d8177848cbd8e3fce, daughterB=44855eaf1c1cbb0be3a6c7e3ed8bb286 in 1.1050 sec 2023-07-24 07:55:05,476 INFO [RS:0;jenkins-hbase4:46877-longCompactions-0] throttle.PressureAwareThroughputController(145): 6159e36da8f6c73d8177848cbd8e3fce#cf2#compaction#124 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:55:05,476 INFO [RS:0;jenkins-hbase4:46877-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6159e36da8f6c73d8177848cbd8e3fce#cf1#compaction#125 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:55:05,490 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/.tmp/cf2/9fe1f407e5664bc383efdcca539d5954 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/9fe1f407e5664bc383efdcca539d5954 2023-07-24 07:55:05,490 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/.tmp/cf1/e017022c2c62412fa5388fe615dec73f as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/e017022c2c62412fa5388fe615dec73f 2023-07-24 07:55:05,494 INFO [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6159e36da8f6c73d8177848cbd8e3fce/cf2 of 6159e36da8f6c73d8177848cbd8e3fce into 9fe1f407e5664bc383efdcca539d5954(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:55:05,494 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6159e36da8f6c73d8177848cbd8e3fce: 2023-07-24 07:55:05,494 INFO [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce., storeName=6159e36da8f6c73d8177848cbd8e3fce/cf2, priority=9, startTime=1690185305456; duration=0sec 2023-07-24 07:55:05,494 DEBUG [RS:0;jenkins-hbase4:46877-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:05,495 INFO [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6159e36da8f6c73d8177848cbd8e3fce/cf1 of 6159e36da8f6c73d8177848cbd8e3fce into e017022c2c62412fa5388fe615dec73f(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:55:05,495 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6159e36da8f6c73d8177848cbd8e3fce: 2023-07-24 07:55:05,495 INFO [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce., storeName=6159e36da8f6c73d8177848cbd8e3fce/cf1, priority=9, startTime=1690185305455; duration=0sec 2023-07-24 07:55:05,495 DEBUG [RS:0;jenkins-hbase4:46877-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:05,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46877] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:60918 deadline: 1690185365610, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4. is not online on jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:05,720 DEBUG [Listener at localhost/42537] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-07-24 07:55:05,723 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:55:05,724 INFO [Listener at localhost/42537] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-07-24 07:55:05,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-07-24 07:55:05,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 07:55:05,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-24 07:55:05,728 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185305727"}]},"ts":"1690185305727"} 2023-07-24 07:55:05,728 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-07-24 07:55:05,730 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-07-24 07:55:05,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6159e36da8f6c73d8177848cbd8e3fce, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=44855eaf1c1cbb0be3a6c7e3ed8bb286, UNASSIGN}] 2023-07-24 07:55:05,732 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=44855eaf1c1cbb0be3a6c7e3ed8bb286, UNASSIGN 2023-07-24 07:55:05,732 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=44855eaf1c1cbb0be3a6c7e3ed8bb286, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:05,732 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690185305732"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185305732"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185305732"}]},"ts":"1690185305732"} 2023-07-24 07:55:05,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 44855eaf1c1cbb0be3a6c7e3ed8bb286, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:55:05,735 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6159e36da8f6c73d8177848cbd8e3fce, UNASSIGN 2023-07-24 07:55:05,735 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=6159e36da8f6c73d8177848cbd8e3fce, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:05,736 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690185305735"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185305735"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185305735"}]},"ts":"1690185305735"} 2023-07-24 07:55:05,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 6159e36da8f6c73d8177848cbd8e3fce, server=jenkins-hbase4.apache.org,46877,1690185199045}] 2023-07-24 07:55:05,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-24 07:55:05,885 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:05,885 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6159e36da8f6c73d8177848cbd8e3fce, disabling compactions & flushes 2023-07-24 07:55:05,885 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce. 2023-07-24 07:55:05,885 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce. 2023-07-24 07:55:05,885 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce. after waiting 0 ms 2023-07-24 07:55:05,885 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce. 2023-07-24 07:55:05,886 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cfbbd450c7794f6fa3128ac2c9c765ea, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cc695c8e1d704c40ba02dde83270ee37, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-23399b5b9cff420e8895f747f5ec5459, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-9587d916339845b48d6533ad1cc97b18, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-be9823364d854b7a929cc9908d81d349, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5e8e995648c8441da272db6122f183e9, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-e4f68f702ae94e14b25162e48ca9ea7f] to archive 2023-07-24 07:55:05,887 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:55:05,888 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cfbbd450c7794f6fa3128ac2c9c765ea to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cfbbd450c7794f6fa3128ac2c9c765ea 2023-07-24 07:55:05,889 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cc695c8e1d704c40ba02dde83270ee37 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-cc695c8e1d704c40ba02dde83270ee37 2023-07-24 07:55:05,890 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-23399b5b9cff420e8895f747f5ec5459 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-23399b5b9cff420e8895f747f5ec5459 2023-07-24 07:55:05,891 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-9587d916339845b48d6533ad1cc97b18 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-9587d916339845b48d6533ad1cc97b18 2023-07-24 07:55:05,892 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-be9823364d854b7a929cc9908d81d349 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-be9823364d854b7a929cc9908d81d349 2023-07-24 07:55:05,893 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5e8e995648c8441da272db6122f183e9 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5e8e995648c8441da272db6122f183e9 2023-07-24 07:55:05,894 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-e4f68f702ae94e14b25162e48ca9ea7f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-e4f68f702ae94e14b25162e48ca9ea7f 2023-07-24 07:55:05,895 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-6479114b8e44449c9ea70d6f169bffbb, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-645a11ffca154d47b5c8dd43599d74b5, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-93d29de62f4347929332b3edf808ce09, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5bb1c3306ce74278878ca12a3acb8af0, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-0d6c9e60733846d79d00324bb180262b, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-b14448b285f047ed8f6c4596dccb7675, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-18b9b7dc95134b0da428d14465b6d07d] to archive 2023-07-24 07:55:05,896 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:55:05,897 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-6479114b8e44449c9ea70d6f169bffbb to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-6479114b8e44449c9ea70d6f169bffbb 2023-07-24 07:55:05,898 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-645a11ffca154d47b5c8dd43599d74b5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-645a11ffca154d47b5c8dd43599d74b5 2023-07-24 07:55:05,899 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-93d29de62f4347929332b3edf808ce09 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-93d29de62f4347929332b3edf808ce09 2023-07-24 07:55:05,900 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5bb1c3306ce74278878ca12a3acb8af0 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-5bb1c3306ce74278878ca12a3acb8af0 2023-07-24 07:55:05,901 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-0d6c9e60733846d79d00324bb180262b to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-0d6c9e60733846d79d00324bb180262b 2023-07-24 07:55:05,902 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-b14448b285f047ed8f6c4596dccb7675 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-b14448b285f047ed8f6c4596dccb7675 2023-07-24 07:55:05,903 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-18b9b7dc95134b0da428d14465b6d07d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/testSplitTableRegionEmptyDaughter=7cbead548a1582f81b9490192fcb37a4-18b9b7dc95134b0da428d14465b6d07d 2023-07-24 07:55:05,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 07:55:05,906 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:55:05,906 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce. 2023-07-24 07:55:05,907 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6159e36da8f6c73d8177848cbd8e3fce: 2023-07-24 07:55:05,908 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:05,908 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:05,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 44855eaf1c1cbb0be3a6c7e3ed8bb286, disabling compactions & flushes 2023-07-24 07:55:05,909 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286. 2023-07-24 07:55:05,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286. 2023-07-24 07:55:05,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286. after waiting 0 ms 2023-07-24 07:55:05,909 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286. 2023-07-24 07:55:05,909 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=6159e36da8f6c73d8177848cbd8e3fce, regionState=CLOSED 2023-07-24 07:55:05,909 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690185305909"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185305909"}]},"ts":"1690185305909"} 2023-07-24 07:55:05,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-07-24 07:55:05,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 6159e36da8f6c73d8177848cbd8e3fce, server=jenkins-hbase4.apache.org,46877,1690185199045 in 174 msec 2023-07-24 07:55:05,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-07-24 07:55:05,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:55:05,912 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286. 2023-07-24 07:55:05,912 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 44855eaf1c1cbb0be3a6c7e3ed8bb286: 2023-07-24 07:55:05,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=6159e36da8f6c73d8177848cbd8e3fce, UNASSIGN in 180 msec 2023-07-24 07:55:05,913 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:05,913 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=44855eaf1c1cbb0be3a6c7e3ed8bb286, regionState=CLOSED 2023-07-24 07:55:05,913 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690185305913"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185305913"}]},"ts":"1690185305913"} 2023-07-24 07:55:05,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-07-24 07:55:05,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 44855eaf1c1cbb0be3a6c7e3ed8bb286, server=jenkins-hbase4.apache.org,46877,1690185199045 in 181 msec 2023-07-24 07:55:05,916 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-07-24 07:55:05,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=44855eaf1c1cbb0be3a6c7e3ed8bb286, UNASSIGN in 184 msec 2023-07-24 07:55:05,917 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185305917"}]},"ts":"1690185305917"} 2023-07-24 07:55:05,918 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-07-24 07:55:05,919 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-07-24 07:55:05,921 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 195 msec 2023-07-24 07:55:06,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-24 07:55:06,030 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-07-24 07:55:06,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-07-24 07:55:06,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 07:55:06,033 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 07:55:06,034 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 07:55:06,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-07-24 07:55:06,038 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:06,038 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:06,038 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:06,040 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/recovered.edits] 2023-07-24 07:55:06,040 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/recovered.edits] 2023-07-24 07:55:06,040 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286/recovered.edits] 2023-07-24 07:55:06,043 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/e017022c2c62412fa5388fe615dec73f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf1/e017022c2c62412fa5388fe615dec73f 2023-07-24 07:55:06,043 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/23399b5b9cff420e8895f747f5ec5459 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/23399b5b9cff420e8895f747f5ec5459 2023-07-24 07:55:06,044 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/5e8e995648c8441da272db6122f183e9 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/5e8e995648c8441da272db6122f183e9 2023-07-24 07:55:06,046 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/9fe1f407e5664bc383efdcca539d5954 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/cf2/9fe1f407e5664bc383efdcca539d5954 2023-07-24 07:55:06,046 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/9587d916339845b48d6533ad1cc97b18 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/9587d916339845b48d6533ad1cc97b18 2023-07-24 07:55:06,046 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286/recovered.edits/88.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286/recovered.edits/88.seqid 2023-07-24 07:55:06,047 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/44855eaf1c1cbb0be3a6c7e3ed8bb286 2023-07-24 07:55:06,047 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/be9823364d854b7a929cc9908d81d349 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/be9823364d854b7a929cc9908d81d349 2023-07-24 07:55:06,048 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/recovered.edits/90.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce/recovered.edits/90.seqid 2023-07-24 07:55:06,048 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cc695c8e1d704c40ba02dde83270ee37 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cc695c8e1d704c40ba02dde83270ee37 2023-07-24 07:55:06,049 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/6159e36da8f6c73d8177848cbd8e3fce 2023-07-24 07:55:06,050 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cfbbd450c7794f6fa3128ac2c9c765ea to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/cfbbd450c7794f6fa3128ac2c9c765ea 2023-07-24 07:55:06,051 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/e4f68f702ae94e14b25162e48ca9ea7f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf1/e4f68f702ae94e14b25162e48ca9ea7f 2023-07-24 07:55:06,053 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/0d6c9e60733846d79d00324bb180262b to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/0d6c9e60733846d79d00324bb180262b 2023-07-24 07:55:06,054 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/18b9b7dc95134b0da428d14465b6d07d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/18b9b7dc95134b0da428d14465b6d07d 2023-07-24 07:55:06,055 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/5bb1c3306ce74278878ca12a3acb8af0 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/5bb1c3306ce74278878ca12a3acb8af0 2023-07-24 07:55:06,056 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/645a11ffca154d47b5c8dd43599d74b5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/645a11ffca154d47b5c8dd43599d74b5 2023-07-24 07:55:06,057 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/6479114b8e44449c9ea70d6f169bffbb to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/6479114b8e44449c9ea70d6f169bffbb 2023-07-24 07:55:06,058 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/93d29de62f4347929332b3edf808ce09 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/93d29de62f4347929332b3edf808ce09 2023-07-24 07:55:06,059 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/b14448b285f047ed8f6c4596dccb7675 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/cf2/b14448b285f047ed8f6c4596dccb7675 2023-07-24 07:55:06,061 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/recovered.edits/85.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4/recovered.edits/85.seqid 2023-07-24 07:55:06,061 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegionEmptyDaughter/7cbead548a1582f81b9490192fcb37a4 2023-07-24 07:55:06,061 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-07-24 07:55:06,063 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 07:55:06,065 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-07-24 07:55:06,067 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-07-24 07:55:06,068 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 07:55:06,068 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-07-24 07:55:06,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185306068"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:06,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185306068"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:06,068 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185306068"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:06,070 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 07:55:06,070 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7cbead548a1582f81b9490192fcb37a4, NAME => 'testSplitTableRegionEmptyDaughter,,1690185300784.7cbead548a1582f81b9490192fcb37a4.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6159e36da8f6c73d8177848cbd8e3fce, NAME => 'testSplitTableRegionEmptyDaughter,,1690185304358.6159e36da8f6c73d8177848cbd8e3fce.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 44855eaf1c1cbb0be3a6c7e3ed8bb286, NAME => 'testSplitTableRegionEmptyDaughter,71,1690185304358.44855eaf1c1cbb0be3a6c7e3ed8bb286.', STARTKEY => '71', ENDKEY => ''}] 2023-07-24 07:55:06,070 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-07-24 07:55:06,070 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690185306070"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:06,072 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-07-24 07:55:06,074 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 07:55:06,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 43 msec 2023-07-24 07:55:06,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-07-24 07:55:06,135 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-07-24 07:55:06,149 INFO [Listener at localhost/42537] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=376 (was 378), OpenFileDescriptor=833 (was 835), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=153 (was 157), ProcessCount=175 (was 175), AvailableMemoryMB=8839 (was 8838) - AvailableMemoryMB LEAK? - 2023-07-24 07:55:06,163 INFO [Listener at localhost/42537] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=376, OpenFileDescriptor=833, MaxFileDescriptor=60000, SystemLoadAverage=153, ProcessCount=175, AvailableMemoryMB=8838 2023-07-24 07:55:06,164 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:55:06,166 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 07:55:06,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 07:55:06,167 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-07-24 07:55:06,169 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 07:55:06,169 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 07:55:06,171 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:06,171 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f empty. 2023-07-24 07:55:06,171 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:06,171 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-07-24 07:55:06,180 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-07-24 07:55:06,182 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => b2a1ba059d990b082f83b8ce7188569f, NAME => 'testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp 2023-07-24 07:55:06,597 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:55:06,597 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing b2a1ba059d990b082f83b8ce7188569f, disabling compactions & flushes 2023-07-24 07:55:06,597 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:06,597 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:06,597 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. after waiting 0 ms 2023-07-24 07:55:06,597 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:06,597 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:06,597 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for b2a1ba059d990b082f83b8ce7188569f: 2023-07-24 07:55:06,600 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 07:55:06,600 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185306600"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185306600"}]},"ts":"1690185306600"} 2023-07-24 07:55:06,601 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 07:55:06,602 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 07:55:06,602 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185306602"}]},"ts":"1690185306602"} 2023-07-24 07:55:06,603 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-07-24 07:55:06,607 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 07:55:06,608 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 07:55:06,608 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 07:55:06,608 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 07:55:06,608 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 07:55:06,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=b2a1ba059d990b082f83b8ce7188569f, ASSIGN}] 2023-07-24 07:55:06,610 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=b2a1ba059d990b082f83b8ce7188569f, ASSIGN 2023-07-24 07:55:06,610 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=b2a1ba059d990b082f83b8ce7188569f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:55:06,611 ERROR [BP-80517894-172.31.14.131-1690185194253 heartbeating to localhost/127.0.0.1:34211] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-80517894-172.31.14.131-1690185194253 (Datanode Uuid 1d2b2de7-56a0-45bb-9a79-d5935e59fefd) service to localhost/127.0.0.1:34211 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-24 07:55:06,761 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 07:55:06,762 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=b2a1ba059d990b082f83b8ce7188569f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:06,762 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185306762"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185306762"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185306762"}]},"ts":"1690185306762"} 2023-07-24 07:55:06,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure b2a1ba059d990b082f83b8ce7188569f, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:55:06,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:06,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b2a1ba059d990b082f83b8ce7188569f, NAME => 'testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.', STARTKEY => '', ENDKEY => ''} 2023-07-24 07:55:06,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:55:06,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:06,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:55:06,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:06,918 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:06,920 INFO [StoreOpener-b2a1ba059d990b082f83b8ce7188569f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:06,921 DEBUG [StoreOpener-b2a1ba059d990b082f83b8ce7188569f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1 2023-07-24 07:55:06,921 DEBUG [StoreOpener-b2a1ba059d990b082f83b8ce7188569f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1 2023-07-24 07:55:06,922 INFO [StoreOpener-b2a1ba059d990b082f83b8ce7188569f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b2a1ba059d990b082f83b8ce7188569f columnFamilyName cf1 2023-07-24 07:55:06,922 INFO [StoreOpener-b2a1ba059d990b082f83b8ce7188569f-1] regionserver.HStore(310): Store=b2a1ba059d990b082f83b8ce7188569f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:06,922 INFO [StoreOpener-b2a1ba059d990b082f83b8ce7188569f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:06,923 DEBUG [StoreOpener-b2a1ba059d990b082f83b8ce7188569f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2 2023-07-24 07:55:06,923 DEBUG [StoreOpener-b2a1ba059d990b082f83b8ce7188569f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2 2023-07-24 07:55:06,923 INFO [StoreOpener-b2a1ba059d990b082f83b8ce7188569f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b2a1ba059d990b082f83b8ce7188569f columnFamilyName cf2 2023-07-24 07:55:06,924 INFO [StoreOpener-b2a1ba059d990b082f83b8ce7188569f-1] regionserver.HStore(310): Store=b2a1ba059d990b082f83b8ce7188569f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:06,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:06,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:06,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:55:06,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:06,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 07:55:06,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b2a1ba059d990b082f83b8ce7188569f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66285493, jitterRate=-0.012269183993339539}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:55:06,929 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b2a1ba059d990b082f83b8ce7188569f: 2023-07-24 07:55:06,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f., pid=143, masterSystemTime=1690185306915 2023-07-24 07:55:06,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:06,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:06,931 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=b2a1ba059d990b082f83b8ce7188569f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:06,931 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185306931"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185306931"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185306931"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185306931"}]},"ts":"1690185306931"} 2023-07-24 07:55:06,933 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-07-24 07:55:06,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure b2a1ba059d990b082f83b8ce7188569f, server=jenkins-hbase4.apache.org,46543,1690185199091 in 169 msec 2023-07-24 07:55:06,935 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-07-24 07:55:06,935 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=b2a1ba059d990b082f83b8ce7188569f, ASSIGN in 325 msec 2023-07-24 07:55:06,935 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 07:55:06,935 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185306935"}]},"ts":"1690185306935"} 2023-07-24 07:55:06,936 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-07-24 07:55:06,938 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 07:55:06,939 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 771 msec 2023-07-24 07:55:07,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:07,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 07:55:07,176 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 07:55:07,176 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,176 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 07:55:07,176 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 07:55:07,177 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,177 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,178 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,178 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,178 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,178 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,179 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,179 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,180 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,180 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,180 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,180 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,180 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,180 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,180 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. started... 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,180 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,181 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b2a1ba059d990b082f83b8ce7188569f 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 07:55:07,181 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,181 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,184 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,184 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,185 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,185 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,185 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,185 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,190 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/2d7101953fa24716b00c109075020382 2023-07-24 07:55:07,201 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/941ac13c793b4e59a2be5e0790287dfd 2023-07-24 07:55:07,205 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/2d7101953fa24716b00c109075020382 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382 2023-07-24 07:55:07,208 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:55:07,209 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/941ac13c793b4e59a2be5e0790287dfd as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd 2023-07-24 07:55:07,212 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:55:07,213 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for b2a1ba059d990b082f83b8ce7188569f in 32ms, sequenceid=6, compaction requested=false 2023-07-24 07:55:07,213 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-07-24 07:55:07,213 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b2a1ba059d990b082f83b8ce7188569f: 2023-07-24 07:55:07,213 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:07,213 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:07,213 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:07,213 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,213 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,213 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,215 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,215 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,215 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,215 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 07:55:07,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35900e32[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:07,217 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 07:55:07,217 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,218 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,218 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,218 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,218 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,219 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,218 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,219 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:07,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,219 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:07,219 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,219 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,219 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,219 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,219 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:07,219 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:07,220 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,220 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,220 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,220 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,224 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,224 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,224 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,224 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,224 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,224 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:55:07,227 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 07:55:07,227 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 07:55:07,227 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:07,227 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:07,227 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 07:55:07,227 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:07,229 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,229 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,229 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,229 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,229 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,229 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,229 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,229 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,230 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,230 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,232 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,232 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,232 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,235 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,235 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,235 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,235 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,235 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 07:55:07,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:07,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,235 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,235 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,235 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,235 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:07,236 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:07,235 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,236 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,235 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,236 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,236 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,236 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,236 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,236 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,236 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,236 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,336 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:07,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:07,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:07,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-24 07:55:07,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 07:55:07,348 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 07:55:07,348 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,348 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 07:55:07,348 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 07:55:07,349 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,349 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,350 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,350 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,350 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,350 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,350 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,350 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:07,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,351 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,351 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,351 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,352 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,352 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,352 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,352 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,352 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,352 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,352 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,352 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,352 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,352 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,353 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:07,353 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. started... 2023-07-24 07:55:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,353 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,353 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b2a1ba059d990b082f83b8ce7188569f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:55:07,355 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,355 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,355 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,356 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,356 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,356 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,365 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/333b0b31ef794c77804ff711a8cd4498 2023-07-24 07:55:07,376 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/26e85e736d6e464d92fd49ac56b1309d 2023-07-24 07:55:07,380 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/333b0b31ef794c77804ff711a8cd4498 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498 2023-07-24 07:55:07,383 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:55:07,384 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/26e85e736d6e464d92fd49ac56b1309d as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d 2023-07-24 07:55:07,387 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 07:55:07,388 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b2a1ba059d990b082f83b8ce7188569f in 35ms, sequenceid=19, compaction requested=false 2023-07-24 07:55:07,389 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b2a1ba059d990b082f83b8ce7188569f: 2023-07-24 07:55:07,389 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:07,389 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:07,389 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:07,389 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,389 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,389 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,391 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,391 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,391 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,391 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 07:55:07,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22c97d37[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:07,393 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 07:55:07,393 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,394 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,394 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,394 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,394 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,394 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,394 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,394 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:07,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,394 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:07,395 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:07,397 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,397 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,397 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,397 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,398 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,398 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,398 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,399 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,399 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,399 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:55:07,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 07:55:07,401 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 07:55:07,401 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:07,401 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:07,401 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 07:55:07,401 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:07,403 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,403 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,403 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,403 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,403 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,403 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,404 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,404 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,404 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,405 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,406 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,406 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,407 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,411 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 07:55:07,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:07,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:07,411 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,412 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,412 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:07,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,412 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,412 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,412 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,512 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:07,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:07,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:07,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-24 07:55:07,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 07:55:07,525 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 07:55:07,525 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,525 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 07:55:07,525 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 07:55:07,526 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,526 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,527 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,527 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,527 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,527 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,527 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,527 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:07,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,528 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,528 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,529 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,529 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,529 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,529 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,529 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,529 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:07,529 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:07,530 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:07,530 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. started... 2023-07-24 07:55:07,530 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b2a1ba059d990b082f83b8ce7188569f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:55:07,531 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,536 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,536 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,536 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,536 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,536 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,543 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/dbc0f0c916b6430eac0e47aafaf8284e 2023-07-24 07:55:07,554 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/f24237e19a344b6ab685bf9b0208b118 2023-07-24 07:55:07,557 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/dbc0f0c916b6430eac0e47aafaf8284e as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e 2023-07-24 07:55:07,560 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:55:07,561 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/f24237e19a344b6ab685bf9b0208b118 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118 2023-07-24 07:55:07,564 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 07:55:07,565 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b2a1ba059d990b082f83b8ce7188569f in 35ms, sequenceid=32, compaction requested=true 2023-07-24 07:55:07,565 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b2a1ba059d990b082f83b8ce7188569f: 2023-07-24 07:55:07,565 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:07,565 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:07,565 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:07,565 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,565 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,565 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,567 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,567 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,567 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,568 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,569 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 07:55:07,569 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f01b21f[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:07,569 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 07:55:07,569 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,570 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,570 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,570 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,571 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,571 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,571 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,572 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,572 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,573 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:55:07,576 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 07:55:07,576 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 07:55:07,576 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:07,576 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:07,576 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 07:55:07,576 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:07,581 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,581 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,581 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,581 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,581 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,581 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,582 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,582 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,582 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,583 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,584 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,584 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,585 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,588 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 07:55:07,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,588 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,589 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:07,589 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:07,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,689 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:07,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:07,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:07,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-24 07:55:07,702 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 07:55:07,702 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 07:55:07,702 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,702 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 07:55:07,702 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 07:55:07,702 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,703 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,704 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,704 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,704 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,704 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,705 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,705 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:07,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,707 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,707 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,707 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,707 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,707 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,707 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:07,707 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:07,707 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:07,707 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. started... 2023-07-24 07:55:07,707 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b2a1ba059d990b082f83b8ce7188569f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:55:07,708 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,710 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,710 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,710 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,710 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,711 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,716 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/7d1d17f1851f45f587bd0353d3010acd 2023-07-24 07:55:07,726 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/1dde980e4ed8427e89d5a1228ffefa52 2023-07-24 07:55:07,729 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/7d1d17f1851f45f587bd0353d3010acd as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd 2023-07-24 07:55:07,733 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:55:07,734 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/1dde980e4ed8427e89d5a1228ffefa52 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52 2023-07-24 07:55:07,737 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 07:55:07,737 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b2a1ba059d990b082f83b8ce7188569f in 30ms, sequenceid=45, compaction requested=true 2023-07-24 07:55:07,737 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b2a1ba059d990b082f83b8ce7188569f: 2023-07-24 07:55:07,737 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:07,738 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:07,738 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:07,738 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,738 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,738 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,740 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,740 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,740 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,740 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 07:55:07,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4363bc74[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:07,742 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 07:55:07,742 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,743 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,743 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,743 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,743 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,743 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:07,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,743 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:07,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:07,744 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:07,745 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,745 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,745 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,747 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,747 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,747 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,747 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,748 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:07,748 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,748 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:07,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:55:07,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 07:55:07,752 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 07:55:07,752 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:07,752 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:07,752 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 07:55:07,752 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:07,754 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,754 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,755 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:07,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:07,755 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,755 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,755 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,755 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:07,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,756 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,756 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,757 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:07,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:07,759 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:07,760 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,762 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,769 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,769 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,769 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,769 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 07:55:07,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:07,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,769 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,769 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:07,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,769 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:07,769 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,770 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:07,769 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,770 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:07,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,770 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,770 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,770 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,770 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:07,770 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,771 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:07,771 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,771 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,771 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:07,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:07,870 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:07,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:07,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:07,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-24 07:55:07,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 07:55:07,883 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 07:55:07,884 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,884 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 07:55:07,884 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 07:55:07,884 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,884 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,885 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,885 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,885 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:07,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,885 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:07,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:07,886 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:07,886 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:07,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:07,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:07,887 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:07,888 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:07,888 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:07,888 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:07,888 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:07,888 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. started... 2023-07-24 07:55:07,888 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b2a1ba059d990b082f83b8ce7188569f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:55:07,889 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,890 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,890 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,890 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:07,890 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:07,890 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:08,298 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed 2023-07-24 07:55:08,310 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/ff98ddff5fc94005846f13919888b56f 2023-07-24 07:55:08,313 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed 2023-07-24 07:55:08,316 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:55:08,317 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/ff98ddff5fc94005846f13919888b56f as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f 2023-07-24 07:55:08,320 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 07:55:08,321 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b2a1ba059d990b082f83b8ce7188569f in 433ms, sequenceid=58, compaction requested=true 2023-07-24 07:55:08,321 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b2a1ba059d990b082f83b8ce7188569f: 2023-07-24 07:55:08,321 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:08,321 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:08,321 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:08,321 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,321 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:08,321 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:08,323 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,323 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:08,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:08,324 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,324 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:08,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:08,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:08,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:08,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 07:55:08,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@dfdf82a[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:08,325 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 07:55:08,326 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,327 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,327 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,327 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,327 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:08,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,327 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:08,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,327 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:08,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,327 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,327 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:08,327 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:08,327 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:08,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,327 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:08,327 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:08,328 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:08,328 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:08,328 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:08,328 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:08,328 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:08,329 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:08,329 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,329 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:08,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:08,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:08,329 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:08,331 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:08,331 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:08,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:08,331 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:08,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:08,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:08,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:55:08,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 07:55:08,333 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 07:55:08,333 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:08,333 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:08,333 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 07:55:08,333 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:08,335 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,335 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:08,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:08,335 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,335 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,335 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,335 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,335 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,336 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,339 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,339 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:08,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,340 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:08,340 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,341 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,343 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:08,343 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:08,344 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:08,343 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:08,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 07:55:08,344 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,344 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:08,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:08,344 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:08,344 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:08,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:08,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:08,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:08,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:08,344 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,344 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,344 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,344 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:08,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,344 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:08,344 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:08,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,345 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,345 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,345 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,345 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,345 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,345 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,445 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:08,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:08,457 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-07-24 07:55:08,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-24 07:55:08,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 07:55:08,459 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 07:55:08,459 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:08,459 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 07:55:08,459 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 07:55:08,459 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,459 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:08,461 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:08,461 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:08,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:08,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:08,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:08,461 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,461 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:08,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:08,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:08,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:08,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:08,461 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,462 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 07:55:08,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:08,463 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:08,462 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 07:55:08,463 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:08,463 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:08,463 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:08,463 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:08,463 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:08,463 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:08,463 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:08,463 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:08,463 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:08,463 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 07:55:08,463 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 07:55:08,463 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 07:55:08,463 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:08,464 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:08,464 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37809,1690185199136' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:08,463 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 07:55:08,463 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:08,464 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 07:55:08,464 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 07:55:08,464 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. started... 2023-07-24 07:55:08,464 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 07:55:08,464 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:08,464 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46877,1690185199045' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:08,464 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b2a1ba059d990b082f83b8ce7188569f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 07:55:08,467 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:08,473 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5 2023-07-24 07:55:08,484 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/c0f26b1f07694e87a07f6743c9f9a869 2023-07-24 07:55:08,487 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5 2023-07-24 07:55:08,490 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:55:08,491 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/c0f26b1f07694e87a07f6743c9f9a869 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869 2023-07-24 07:55:08,494 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 07:55:08,495 INFO [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for b2a1ba059d990b082f83b8ce7188569f in 31ms, sequenceid=71, compaction requested=true 2023-07-24 07:55:08,495 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b2a1ba059d990b082f83b8ce7188569f: 2023-07-24 07:55:08,495 DEBUG [rs(jenkins-hbase4.apache.org,46543,1690185199091)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:08,495 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 07:55:08,495 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 07:55:08,495 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,495 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 07:55:08,495 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 07:55:08,497 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,497 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:08,497 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,497 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 07:55:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:08,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:08,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,46543,1690185199091' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 07:55:08,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53a365fd[Count = 0] remaining members to acquire global barrier 2023-07-24 07:55:08,499 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 07:55:08,499 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,500 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,500 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,500 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:08,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46543,1690185199091' in zk 2023-07-24 07:55:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:08,501 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,501 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 07:55:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,46877,1690185199045' in zk 2023-07-24 07:55:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 07:55:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 07:55:08,501 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37809,1690185199136' in zk 2023-07-24 07:55:08,505 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,505 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:08,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:08,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:08,505 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:08,506 DEBUG [member: 'jenkins-hbase4.apache.org,46543,1690185199091' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:08,506 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:08,506 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:08,506 DEBUG [member: 'jenkins-hbase4.apache.org,46877,1690185199045' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:08,507 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 07:55:08,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:08,507 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:08,507 DEBUG [member: 'jenkins-hbase4.apache.org,37809,1690185199136' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 07:55:08,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:08,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:08,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,46543,1690185199091': 2023-07-24 07:55:08,511 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,46543,1690185199091' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 07:55:08,511 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 07:55:08,511 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 07:55:08,511 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 07:55:08,511 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 07:55:08,511 INFO [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 07:55:08,514 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,514 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 07:55:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 07:55:08,514 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,514 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,514 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,514 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,514 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 07:55:08,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,515 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,515 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 07:55:08,516 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 07:55:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 07:55:08,518 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,518 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,518 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,523 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,523 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:08,523 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:08,523 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:08,523 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 07:55:08,523 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:08,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:08,523 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 07:55:08,523 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 07:55:08,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 07:55:08,523 DEBUG [(jenkins-hbase4.apache.org,37123,1690185197841)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 07:55:08,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,523 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 07:55:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 07:55:08,523 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,524 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 07:55:08,524 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 07:55:08,524 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 07:55:08,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 07:55:08,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,524 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:08,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 07:55:08,524 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,524 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:08,524 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,524 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 07:55:08,524 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 07:55:08,624 DEBUG [Listener at localhost/42537] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 07:55:08,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 07:55:08,632 DEBUG [Listener at localhost/42537] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,633 DEBUG [Listener at localhost/42537] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=b2a1ba059d990b082f83b8ce7188569f, daughterA=06243f0beecd83ce6d9ba0205da378bb, daughterB=2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:08,634 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=b2a1ba059d990b082f83b8ce7188569f, daughterA=06243f0beecd83ce6d9ba0205da378bb, daughterB=2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:08,634 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=b2a1ba059d990b082f83b8ce7188569f, daughterA=06243f0beecd83ce6d9ba0205da378bb, daughterB=2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:08,634 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=b2a1ba059d990b082f83b8ce7188569f, daughterA=06243f0beecd83ce6d9ba0205da378bb, daughterB=2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:08,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=b2a1ba059d990b082f83b8ce7188569f, UNASSIGN}] 2023-07-24 07:55:08,636 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=b2a1ba059d990b082f83b8ce7188569f, UNASSIGN 2023-07-24 07:55:08,637 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=b2a1ba059d990b082f83b8ce7188569f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:08,637 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185308637"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185308637"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185308637"}]},"ts":"1690185308637"} 2023-07-24 07:55:08,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure b2a1ba059d990b082f83b8ce7188569f, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:55:08,961 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:08,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b2a1ba059d990b082f83b8ce7188569f, disabling compactions & flushes 2023-07-24 07:55:08,961 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:08,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:08,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. after waiting 0 ms 2023-07-24 07:55:08,961 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:08,961 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing b2a1ba059d990b082f83b8ce7188569f 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 07:55:08,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/280c62fe41e24598b8fa52b9037a9c09 2023-07-24 07:55:08,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/b3723cac00d943afaf8bb18347bdee6a 2023-07-24 07:55:08,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf1/280c62fe41e24598b8fa52b9037a9c09 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09 2023-07-24 07:55:08,990 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:55:08,990 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/.tmp/cf2/b3723cac00d943afaf8bb18347bdee6a as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a 2023-07-24 07:55:08,994 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 07:55:08,995 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for b2a1ba059d990b082f83b8ce7188569f in 34ms, sequenceid=82, compaction requested=true 2023-07-24 07:55:08,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 07:55:08,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:55:08,999 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. 2023-07-24 07:55:08,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b2a1ba059d990b082f83b8ce7188569f: 2023-07-24 07:55:09,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,001 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=b2a1ba059d990b082f83b8ce7188569f, regionState=CLOSED 2023-07-24 07:55:09,001 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185309001"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185309001"}]},"ts":"1690185309001"} 2023-07-24 07:55:09,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-07-24 07:55:09,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure b2a1ba059d990b082f83b8ce7188569f, server=jenkins-hbase4.apache.org,46543,1690185199091 in 364 msec 2023-07-24 07:55:09,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-07-24 07:55:09,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=b2a1ba059d990b082f83b8ce7188569f, UNASSIGN in 367 msec 2023-07-24 07:55:09,013 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=b2a1ba059d990b082f83b8ce7188569f, threads=14 2023-07-24 07:55:09,014 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,014 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,014 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,014 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,014 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,014 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,014 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,014 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,014 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,015 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,015 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,018 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,019 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,019 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,119 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,123 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,141 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,142 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,161 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,181 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,190 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,190 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,194 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,194 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,194 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,195 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,198 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,204 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5 for region: b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:09,204 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region b2a1ba059d990b082f83b8ce7188569f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 07:55:09,238 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:55:09,239 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 07:55:09,241 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690185309241"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1690185309241"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1690185309241"}]},"ts":"1690185309241"} 2023-07-24 07:55:09,241 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185309241"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185309241"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185309241"}]},"ts":"1690185309241"} 2023-07-24 07:55:09,241 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185309241"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185309241"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185309241"}]},"ts":"1690185309241"} 2023-07-24 07:55:09,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=06243f0beecd83ce6d9ba0205da378bb, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=2487195f4a5dba06271bb7d71774ce66, ASSIGN}] 2023-07-24 07:55:09,245 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=2487195f4a5dba06271bb7d71774ce66, ASSIGN 2023-07-24 07:55:09,245 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=2487195f4a5dba06271bb7d71774ce66, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:55:09,246 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=06243f0beecd83ce6d9ba0205da378bb, ASSIGN 2023-07-24 07:55:09,246 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=06243f0beecd83ce6d9ba0205da378bb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,46543,1690185199091; forceNewPlan=false, retain=false 2023-07-24 07:55:09,396 INFO [jenkins-hbase4:37123] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 07:55:09,397 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=2487195f4a5dba06271bb7d71774ce66, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:09,397 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185309397"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185309397"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185309397"}]},"ts":"1690185309397"} 2023-07-24 07:55:09,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=148, state=RUNNABLE; OpenRegionProcedure 2487195f4a5dba06271bb7d71774ce66, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:55:09,401 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=06243f0beecd83ce6d9ba0205da378bb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:09,401 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185309401"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185309401"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185309401"}]},"ts":"1690185309401"} 2023-07-24 07:55:09,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=147, state=RUNNABLE; OpenRegionProcedure 06243f0beecd83ce6d9ba0205da378bb, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:55:09,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. 2023-07-24 07:55:09,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2487195f4a5dba06271bb7d71774ce66, NAME => 'testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.', STARTKEY => '41', ENDKEY => ''} 2023-07-24 07:55:09,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:55:09,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:09,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:55:09,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:09,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:09,555 INFO [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:09,556 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1 2023-07-24 07:55:09,556 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1 2023-07-24 07:55:09,557 INFO [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2487195f4a5dba06271bb7d71774ce66 columnFamilyName cf1 2023-07-24 07:55:09,567 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09-top 2023-07-24 07:55:09,570 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5-top 2023-07-24 07:55:09,574 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382-top 2023-07-24 07:55:09,577 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498-top 2023-07-24 07:55:09,580 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed-top 2023-07-24 07:55:09,583 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd-top 2023-07-24 07:55:09,586 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e-top 2023-07-24 07:55:09,586 INFO [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(310): Store=2487195f4a5dba06271bb7d71774ce66/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:09,587 INFO [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:09,587 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2 2023-07-24 07:55:09,587 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2 2023-07-24 07:55:09,588 INFO [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2487195f4a5dba06271bb7d71774ce66 columnFamilyName cf2 2023-07-24 07:55:09,599 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52-top 2023-07-24 07:55:09,602 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d-top 2023-07-24 07:55:09,605 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd-top 2023-07-24 07:55:09,608 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a-top 2023-07-24 07:55:09,611 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869-top 2023-07-24 07:55:09,614 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118-top 2023-07-24 07:55:09,618 DEBUG [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f-top 2023-07-24 07:55:09,618 INFO [StoreOpener-2487195f4a5dba06271bb7d71774ce66-1] regionserver.HStore(310): Store=2487195f4a5dba06271bb7d71774ce66/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:09,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:09,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:09,621 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:55:09,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:09,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2487195f4a5dba06271bb7d71774ce66; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61892621, jitterRate=-0.07772807776927948}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:55:09,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2487195f4a5dba06271bb7d71774ce66: 2023-07-24 07:55:09,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66., pid=149, masterSystemTime=1690185309550 2023-07-24 07:55:09,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:09,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:09,626 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:55:09,626 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:55:09,626 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. 2023-07-24 07:55:09,626 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. 2023-07-24 07:55:09,626 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1912): 2487195f4a5dba06271bb7d71774ce66/cf2 is initiating minor compaction (all files) 2023-07-24 07:55:09,626 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1912): 2487195f4a5dba06271bb7d71774ce66/cf1 is initiating minor compaction (all files) 2023-07-24 07:55:09,626 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 2487195f4a5dba06271bb7d71774ce66/cf2 in testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. 2023-07-24 07:55:09,626 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 2487195f4a5dba06271bb7d71774ce66/cf1 in testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. 2023-07-24 07:55:09,626 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09-top] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/.tmp, totalSize=34.9 K 2023-07-24 07:55:09,626 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a-top] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/.tmp, totalSize=34.9 K 2023-07-24 07:55:09,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. 2023-07-24 07:55:09,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. 2023-07-24 07:55:09,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. 2023-07-24 07:55:09,627 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690185307171 2023-07-24 07:55:09,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 06243f0beecd83ce6d9ba0205da378bb, NAME => 'testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.', STARTKEY => '', ENDKEY => '41'} 2023-07-24 07:55:09,627 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=2487195f4a5dba06271bb7d71774ce66, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:09,627 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690185307171 2023-07-24 07:55:09,627 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185309627"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185309627"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185309627"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185309627"}]},"ts":"1690185309627"} 2023-07-24 07:55:09,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 07:55:09,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:09,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 07:55:09,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:09,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:09,628 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690185307337 2023-07-24 07:55:09,628 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690185307337 2023-07-24 07:55:09,628 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690185307513 2023-07-24 07:55:09,629 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690185307513 2023-07-24 07:55:09,629 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690185307691 2023-07-24 07:55:09,629 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690185307691 2023-07-24 07:55:09,629 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690185307872 2023-07-24 07:55:09,630 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690185307872 2023-07-24 07:55:09,630 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690185308447 2023-07-24 07:55:09,630 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690185308447 2023-07-24 07:55:09,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=148 2023-07-24 07:55:09,631 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690185308626 2023-07-24 07:55:09,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=148, state=SUCCESS; OpenRegionProcedure 2487195f4a5dba06271bb7d71774ce66, server=jenkins-hbase4.apache.org,46543,1690185199091 in 230 msec 2023-07-24 07:55:09,631 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690185308626 2023-07-24 07:55:09,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=2487195f4a5dba06271bb7d71774ce66, ASSIGN in 387 msec 2023-07-24 07:55:09,636 INFO [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:09,637 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1 2023-07-24 07:55:09,637 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1 2023-07-24 07:55:09,638 INFO [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 06243f0beecd83ce6d9ba0205da378bb columnFamilyName cf1 2023-07-24 07:55:09,641 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] throttle.PressureAwareThroughputController(145): 2487195f4a5dba06271bb7d71774ce66#cf2#compaction#140 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:55:09,641 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] throttle.PressureAwareThroughputController(145): 2487195f4a5dba06271bb7d71774ce66#cf1#compaction#141 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:55:09,662 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09-bottom 2023-07-24 07:55:09,662 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/.tmp/cf1/f9639230f9a34bd1bab65f5878095fce as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/f9639230f9a34bd1bab65f5878095fce 2023-07-24 07:55:09,664 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/.tmp/cf2/86115965ea6b4532b695119b61096e5e as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/86115965ea6b4532b695119b61096e5e 2023-07-24 07:55:09,667 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5-bottom 2023-07-24 07:55:09,668 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2487195f4a5dba06271bb7d71774ce66/cf2 of 2487195f4a5dba06271bb7d71774ce66 into 86115965ea6b4532b695119b61096e5e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:55:09,668 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 2487195f4a5dba06271bb7d71774ce66/cf1 of 2487195f4a5dba06271bb7d71774ce66 into f9639230f9a34bd1bab65f5878095fce(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:55:09,668 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2487195f4a5dba06271bb7d71774ce66: 2023-07-24 07:55:09,668 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 2487195f4a5dba06271bb7d71774ce66: 2023-07-24 07:55:09,668 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66., storeName=2487195f4a5dba06271bb7d71774ce66/cf2, priority=9, startTime=1690185309626; duration=0sec 2023-07-24 07:55:09,668 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66., storeName=2487195f4a5dba06271bb7d71774ce66/cf1, priority=9, startTime=1690185309625; duration=0sec 2023-07-24 07:55:09,668 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:09,668 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:09,670 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382-bottom 2023-07-24 07:55:09,673 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498-bottom 2023-07-24 07:55:09,676 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed-bottom 2023-07-24 07:55:09,679 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd-bottom 2023-07-24 07:55:09,681 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e-bottom 2023-07-24 07:55:09,681 INFO [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(310): Store=06243f0beecd83ce6d9ba0205da378bb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:09,682 INFO [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:09,682 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2 2023-07-24 07:55:09,682 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2 2023-07-24 07:55:09,683 INFO [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 06243f0beecd83ce6d9ba0205da378bb columnFamilyName cf2 2023-07-24 07:55:09,694 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52-bottom 2023-07-24 07:55:09,696 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d-bottom 2023-07-24 07:55:09,699 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd-bottom 2023-07-24 07:55:09,701 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a-bottom 2023-07-24 07:55:09,704 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869-bottom 2023-07-24 07:55:09,707 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118-bottom 2023-07-24 07:55:09,711 DEBUG [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(539): loaded hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f-bottom 2023-07-24 07:55:09,711 INFO [StoreOpener-06243f0beecd83ce6d9ba0205da378bb-1] regionserver.HStore(310): Store=06243f0beecd83ce6d9ba0205da378bb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 07:55:09,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:09,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:09,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 07:55:09,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:09,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 06243f0beecd83ce6d9ba0205da378bb; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63650046, jitterRate=-0.05154040455818176}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 07:55:09,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 06243f0beecd83ce6d9ba0205da378bb: 2023-07-24 07:55:09,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb., pid=150, masterSystemTime=1690185309550 2023-07-24 07:55:09,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:09,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 07:55:09,717 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:55:09,717 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 07:55:09,717 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. 2023-07-24 07:55:09,717 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1912): 06243f0beecd83ce6d9ba0205da378bb/cf1 is initiating minor compaction (all files) 2023-07-24 07:55:09,717 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. 2023-07-24 07:55:09,717 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1912): 06243f0beecd83ce6d9ba0205da378bb/cf2 is initiating minor compaction (all files) 2023-07-24 07:55:09,717 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 06243f0beecd83ce6d9ba0205da378bb/cf1 in testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. 2023-07-24 07:55:09,717 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 06243f0beecd83ce6d9ba0205da378bb/cf2 in testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. 2023-07-24 07:55:09,717 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09-bottom] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/.tmp, totalSize=34.9 K 2023-07-24 07:55:09,717 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a-bottom] into tmpdir=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/.tmp, totalSize=34.9 K 2023-07-24 07:55:09,718 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185307171 2023-07-24 07:55:09,718 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690185307171 2023-07-24 07:55:09,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. 2023-07-24 07:55:09,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. 2023-07-24 07:55:09,718 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185307337 2023-07-24 07:55:09,719 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690185307337 2023-07-24 07:55:09,719 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=06243f0beecd83ce6d9ba0205da378bb, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:09,719 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185307513 2023-07-24 07:55:09,719 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185309719"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690185309719"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690185309719"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690185309719"}]},"ts":"1690185309719"} 2023-07-24 07:55:09,719 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690185307513 2023-07-24 07:55:09,719 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185307691 2023-07-24 07:55:09,719 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting 1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690185307691 2023-07-24 07:55:09,719 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185307872 2023-07-24 07:55:09,720 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690185307872 2023-07-24 07:55:09,720 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185308447 2023-07-24 07:55:09,720 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690185308447 2023-07-24 07:55:09,720 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] compactions.Compactor(207): Compacting 280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185308626 2023-07-24 07:55:09,721 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] compactions.Compactor(207): Compacting b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690185308626 2023-07-24 07:55:09,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=147 2023-07-24 07:55:09,721 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=147, state=SUCCESS; OpenRegionProcedure 06243f0beecd83ce6d9ba0205da378bb, server=jenkins-hbase4.apache.org,46543,1690185199091 in 318 msec 2023-07-24 07:55:09,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-07-24 07:55:09,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=06243f0beecd83ce6d9ba0205da378bb, ASSIGN in 478 msec 2023-07-24 07:55:09,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=b2a1ba059d990b082f83b8ce7188569f, daughterA=06243f0beecd83ce6d9ba0205da378bb, daughterB=2487195f4a5dba06271bb7d71774ce66 in 1.0910 sec 2023-07-24 07:55:09,730 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] throttle.PressureAwareThroughputController(145): 06243f0beecd83ce6d9ba0205da378bb#cf2#compaction#143 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:55:09,730 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] throttle.PressureAwareThroughputController(145): 06243f0beecd83ce6d9ba0205da378bb#cf1#compaction#142 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 07:55:09,743 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/.tmp/cf1/ccca90a2f5a348ce8a0c2a89762a9df5 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/ccca90a2f5a348ce8a0c2a89762a9df5 2023-07-24 07:55:09,743 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/.tmp/cf2/d82e039d36d54958aeca92d95d512287 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/d82e039d36d54958aeca92d95d512287 2023-07-24 07:55:09,749 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 06243f0beecd83ce6d9ba0205da378bb/cf1 of 06243f0beecd83ce6d9ba0205da378bb into ccca90a2f5a348ce8a0c2a89762a9df5(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:55:09,749 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 06243f0beecd83ce6d9ba0205da378bb/cf2 of 06243f0beecd83ce6d9ba0205da378bb into d82e039d36d54958aeca92d95d512287(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 07:55:09,749 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 06243f0beecd83ce6d9ba0205da378bb: 2023-07-24 07:55:09,749 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 06243f0beecd83ce6d9ba0205da378bb: 2023-07-24 07:55:09,749 INFO [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb., storeName=06243f0beecd83ce6d9ba0205da378bb/cf1, priority=9, startTime=1690185309716; duration=0sec 2023-07-24 07:55:09,749 INFO [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb., storeName=06243f0beecd83ce6d9ba0205da378bb/cf2, priority=9, startTime=1690185309717; duration=0sec 2023-07-24 07:55:09,750 DEBUG [RS:1;jenkins-hbase4:46543-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:09,750 DEBUG [RS:1;jenkins-hbase4:46543-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 07:55:09,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46543] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:36420 deadline: 1690185369961, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f. is not online on jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:10,052 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 07:55:10,073 DEBUG [Listener at localhost/42537] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 07:55:10,085 INFO [Listener at localhost/42537] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-24 07:55:10,087 DEBUG [Listener at localhost/42537] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 07:55:10,097 WARN [Listener at localhost/42537] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 07:55:10,099 INFO [Listener at localhost/42537] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-07-24 07:55:10,100 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-07-24 07:55:10,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-07-24 07:55:10,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-24 07:55:10,105 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185310105"}]},"ts":"1690185310105"} 2023-07-24 07:55:10,106 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-07-24 07:55:10,108 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-07-24 07:55:10,109 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=06243f0beecd83ce6d9ba0205da378bb, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=2487195f4a5dba06271bb7d71774ce66, UNASSIGN}] 2023-07-24 07:55:10,110 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=2487195f4a5dba06271bb7d71774ce66, UNASSIGN 2023-07-24 07:55:10,111 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=2487195f4a5dba06271bb7d71774ce66, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:10,111 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185310111"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185310111"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185310111"}]},"ts":"1690185310111"} 2023-07-24 07:55:10,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 2487195f4a5dba06271bb7d71774ce66, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:55:10,115 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=06243f0beecd83ce6d9ba0205da378bb, UNASSIGN 2023-07-24 07:55:10,115 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=06243f0beecd83ce6d9ba0205da378bb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:10,115 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185310115"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690185310115"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690185310115"}]},"ts":"1690185310115"} 2023-07-24 07:55:10,119 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 06243f0beecd83ce6d9ba0205da378bb, server=jenkins-hbase4.apache.org,46543,1690185199091}] 2023-07-24 07:55:10,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-24 07:55:10,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:10,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 06243f0beecd83ce6d9ba0205da378bb, disabling compactions & flushes 2023-07-24 07:55:10,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. 2023-07-24 07:55:10,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. 2023-07-24 07:55:10,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. after waiting 0 ms 2023-07-24 07:55:10,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. 2023-07-24 07:55:10,265 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09-bottom] to archive 2023-07-24 07:55:10,266 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:55:10,268 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,269 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,270 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,271 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,272 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,273 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,274 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,275 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869-bottom, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a-bottom] to archive 2023-07-24 07:55:10,276 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:55:10,277 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,278 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,279 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,280 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,281 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,282 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,283 DEBUG [StoreCloser-testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 07:55:10,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:55:10,288 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb. 2023-07-24 07:55:10,288 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 06243f0beecd83ce6d9ba0205da378bb: 2023-07-24 07:55:10,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:10,289 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:10,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2487195f4a5dba06271bb7d71774ce66, disabling compactions & flushes 2023-07-24 07:55:10,290 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. 2023-07-24 07:55:10,290 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=06243f0beecd83ce6d9ba0205da378bb, regionState=CLOSED 2023-07-24 07:55:10,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. 2023-07-24 07:55:10,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. after waiting 0 ms 2023-07-24 07:55:10,290 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185310290"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185310290"}]},"ts":"1690185310290"} 2023-07-24 07:55:10,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. 2023-07-24 07:55:10,291 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09-top] to archive 2023-07-24 07:55:10,292 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:55:10,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-07-24 07:55:10,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 06243f0beecd83ce6d9ba0205da378bb, server=jenkins-hbase4.apache.org,46543,1690185199091 in 175 msec 2023-07-24 07:55:10,293 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/2d7101953fa24716b00c109075020382.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,293 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=06243f0beecd83ce6d9ba0205da378bb, UNASSIGN in 183 msec 2023-07-24 07:55:10,294 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/333b0b31ef794c77804ff711a8cd4498.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,295 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/dbc0f0c916b6430eac0e47aafaf8284e.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,296 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/7d1d17f1851f45f587bd0353d3010acd.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,297 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,298 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,299 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/280c62fe41e24598b8fa52b9037a9c09.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,300 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869-top, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f->hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a-top] to archive 2023-07-24 07:55:10,301 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 07:55:10,302 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/941ac13c793b4e59a2be5e0790287dfd.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,303 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/26e85e736d6e464d92fd49ac56b1309d.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,304 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/f24237e19a344b6ab685bf9b0208b118.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,305 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/1dde980e4ed8427e89d5a1228ffefa52.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,305 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/ff98ddff5fc94005846f13919888b56f.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,306 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/c0f26b1f07694e87a07f6743c9f9a869.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,307 DEBUG [StoreCloser-testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/b3723cac00d943afaf8bb18347bdee6a.b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,310 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 07:55:10,311 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:55:10,312 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66. 2023-07-24 07:55:10,312 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2487195f4a5dba06271bb7d71774ce66: 2023-07-24 07:55:10,313 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:10,313 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=2487195f4a5dba06271bb7d71774ce66, regionState=CLOSED 2023-07-24 07:55:10,314 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690185310313"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690185310313"}]},"ts":"1690185310313"} 2023-07-24 07:55:10,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-07-24 07:55:10,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 2487195f4a5dba06271bb7d71774ce66, server=jenkins-hbase4.apache.org,46543,1690185199091 in 202 msec 2023-07-24 07:55:10,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-07-24 07:55:10,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=2487195f4a5dba06271bb7d71774ce66, UNASSIGN in 207 msec 2023-07-24 07:55:10,317 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690185310317"}]},"ts":"1690185310317"} 2023-07-24 07:55:10,318 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-07-24 07:55:10,320 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-07-24 07:55:10,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 220 msec 2023-07-24 07:55:10,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-24 07:55:10,407 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-07-24 07:55:10,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-07-24 07:55:10,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 07:55:10,410 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 07:55:10,411 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 07:55:10,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-07-24 07:55:10,415 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,415 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:10,415 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:10,417 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/recovered.edits] 2023-07-24 07:55:10,417 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/recovered.edits] 2023-07-24 07:55:10,417 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2, FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/recovered.edits] 2023-07-24 07:55:10,420 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/ccca90a2f5a348ce8a0c2a89762a9df5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf1/ccca90a2f5a348ce8a0c2a89762a9df5 2023-07-24 07:55:10,420 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/f9639230f9a34bd1bab65f5878095fce to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf1/f9639230f9a34bd1bab65f5878095fce 2023-07-24 07:55:10,420 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/280c62fe41e24598b8fa52b9037a9c09 2023-07-24 07:55:10,422 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2b112ead0f2b4a29bfa45d9bef2cfec5 2023-07-24 07:55:10,422 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/d82e039d36d54958aeca92d95d512287 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/cf2/d82e039d36d54958aeca92d95d512287 2023-07-24 07:55:10,423 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/86115965ea6b4532b695119b61096e5e to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/cf2/86115965ea6b4532b695119b61096e5e 2023-07-24 07:55:10,423 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/2d7101953fa24716b00c109075020382 2023-07-24 07:55:10,425 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/333b0b31ef794c77804ff711a8cd4498 2023-07-24 07:55:10,425 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/recovered.edits/90.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb/recovered.edits/90.seqid 2023-07-24 07:55:10,425 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/06243f0beecd83ce6d9ba0205da378bb 2023-07-24 07:55:10,425 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/recovered.edits/90.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66/recovered.edits/90.seqid 2023-07-24 07:55:10,426 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/6b06fd8f2e0f4dabba9aae9f84a336ed 2023-07-24 07:55:10,426 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/2487195f4a5dba06271bb7d71774ce66 2023-07-24 07:55:10,427 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/7d1d17f1851f45f587bd0353d3010acd 2023-07-24 07:55:10,428 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf1/dbc0f0c916b6430eac0e47aafaf8284e 2023-07-24 07:55:10,430 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/1dde980e4ed8427e89d5a1228ffefa52 2023-07-24 07:55:10,431 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/26e85e736d6e464d92fd49ac56b1309d 2023-07-24 07:55:10,432 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/941ac13c793b4e59a2be5e0790287dfd 2023-07-24 07:55:10,433 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/b3723cac00d943afaf8bb18347bdee6a 2023-07-24 07:55:10,434 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/c0f26b1f07694e87a07f6743c9f9a869 2023-07-24 07:55:10,435 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118 to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/f24237e19a344b6ab685bf9b0208b118 2023-07-24 07:55:10,436 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/cf2/ff98ddff5fc94005846f13919888b56f 2023-07-24 07:55:10,438 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/recovered.edits/85.seqid to hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/archive/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f/recovered.edits/85.seqid 2023-07-24 07:55:10,438 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/.tmp/data/default/testSplitTableRegion/b2a1ba059d990b082f83b8ce7188569f 2023-07-24 07:55:10,438 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-07-24 07:55:10,440 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 07:55:10,442 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-07-24 07:55:10,444 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-07-24 07:55:10,445 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 07:55:10,445 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-07-24 07:55:10,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185310445"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:10,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185310445"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:10,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690185310445"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:10,447 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 07:55:10,447 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => b2a1ba059d990b082f83b8ce7188569f, NAME => 'testSplitTableRegion,,1690185306167.b2a1ba059d990b082f83b8ce7188569f.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 06243f0beecd83ce6d9ba0205da378bb, NAME => 'testSplitTableRegion,,1690185308632.06243f0beecd83ce6d9ba0205da378bb.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 2487195f4a5dba06271bb7d71774ce66, NAME => 'testSplitTableRegion,41,1690185308632.2487195f4a5dba06271bb7d71774ce66.', STARTKEY => '41', ENDKEY => ''}] 2023-07-24 07:55:10,447 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-07-24 07:55:10,447 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690185310447"}]},"ts":"9223372036854775807"} 2023-07-24 07:55:10,448 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-07-24 07:55:10,450 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 07:55:10,451 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 42 msec 2023-07-24 07:55:10,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37123] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-07-24 07:55:10,513 INFO [Listener at localhost/42537] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-07-24 07:55:10,527 INFO [Listener at localhost/42537] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=378 (was 376) - Thread LEAK? -, OpenFileDescriptor=833 (was 833), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=148 (was 153), ProcessCount=175 (was 175), AvailableMemoryMB=8785 (was 8838) 2023-07-24 07:55:10,527 INFO [Listener at localhost/42537] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-24 07:55:10,527 INFO [Listener at localhost/42537] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-24 07:55:10,528 DEBUG [Listener at localhost/42537] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x72621703 to 127.0.0.1:55594 2023-07-24 07:55:10,528 DEBUG [Listener at localhost/42537] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 07:55:10,529 DEBUG [Listener at localhost/42537] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-24 07:55:10,529 DEBUG [Listener at localhost/42537] util.JVMClusterUtil(257): Found active master hash=1735069612, stopped=false 2023-07-24 07:55:10,529 INFO [Listener at localhost/42537] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37123,1690185197841 2023-07-24 07:55:10,531 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 07:55:10,531 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 07:55:10,531 INFO [Listener at localhost/42537] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 07:55:10,531 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 07:55:10,531 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 07:55:10,532 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 07:55:10,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 07:55:10,532 DEBUG [Listener at localhost/42537] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4741057f to 127.0.0.1:55594 2023-07-24 07:55:10,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 07:55:10,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 07:55:10,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 07:55:10,532 DEBUG [Listener at localhost/42537] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 07:55:10,533 INFO [Listener at localhost/42537] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46877,1690185199045' ***** 2023-07-24 07:55:10,533 INFO [Listener at localhost/42537] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 07:55:10,533 INFO [Listener at localhost/42537] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46543,1690185199091' ***** 2023-07-24 07:55:10,533 INFO [Listener at localhost/42537] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 07:55:10,533 INFO [Listener at localhost/42537] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,37809,1690185199136' ***** 2023-07-24 07:55:10,533 INFO [Listener at localhost/42537] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 07:55:10,533 INFO [RS:0;jenkins-hbase4:46877] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 07:55:10,533 INFO [RS:1;jenkins-hbase4:46543] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 07:55:10,533 INFO [RS:1;jenkins-hbase4:46543] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 07:55:10,533 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 07:55:10,533 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 07:55:10,534 INFO [RS:1;jenkins-hbase4:46543] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 07:55:10,533 INFO [RS:2;jenkins-hbase4:37809] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 07:55:10,534 INFO [RS:0;jenkins-hbase4:46877] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 07:55:10,534 INFO [RS:0;jenkins-hbase4:46877] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 07:55:10,534 INFO [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:10,534 DEBUG [RS:0;jenkins-hbase4:46877] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1bc1e74e to 127.0.0.1:55594 2023-07-24 07:55:10,534 DEBUG [RS:0;jenkins-hbase4:46877] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 07:55:10,534 INFO [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46877,1690185199045; all regions closed. 2023-07-24 07:55:10,534 INFO [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:10,535 DEBUG [RS:1;jenkins-hbase4:46543] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0bb93727 to 127.0.0.1:55594 2023-07-24 07:55:10,535 DEBUG [RS:1;jenkins-hbase4:46543] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 07:55:10,534 INFO [RS:2;jenkins-hbase4:37809] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 07:55:10,534 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 07:55:10,535 INFO [RS:2;jenkins-hbase4:37809] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 07:55:10,535 INFO [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46543,1690185199091; all regions closed. 2023-07-24 07:55:10,535 INFO [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(3305): Received CLOSE for 61dcffa18562b861467de5a803476e2c 2023-07-24 07:55:10,536 INFO [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:10,536 DEBUG [RS:2;jenkins-hbase4:37809] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5c963d3d to 127.0.0.1:55594 2023-07-24 07:55:10,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 61dcffa18562b861467de5a803476e2c, disabling compactions & flushes 2023-07-24 07:55:10,536 DEBUG [RS:2;jenkins-hbase4:37809] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 07:55:10,536 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:55:10,536 INFO [RS:2;jenkins-hbase4:37809] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 07:55:10,536 INFO [RS:2;jenkins-hbase4:37809] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 07:55:10,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:55:10,536 INFO [RS:2;jenkins-hbase4:37809] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 07:55:10,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. after waiting 0 ms 2023-07-24 07:55:10,536 INFO [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-24 07:55:10,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:55:10,536 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 61dcffa18562b861467de5a803476e2c 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-24 07:55:10,537 INFO [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-24 07:55:10,537 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1478): Online Regions={61dcffa18562b861467de5a803476e2c=hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c., 1588230740=hbase:meta,,1.1588230740} 2023-07-24 07:55:10,537 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-24 07:55:10,537 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-24 07:55:10,537 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-24 07:55:10,537 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-24 07:55:10,537 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-24 07:55:10,537 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-07-24 07:55:10,538 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1504): Waiting on 1588230740, 61dcffa18562b861467de5a803476e2c 2023-07-24 07:55:10,550 DEBUG [RS:1;jenkins-hbase4:46543] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/oldWALs 2023-07-24 07:55:10,551 INFO [RS:1;jenkins-hbase4:46543] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46543%2C1690185199091:(num 1690185200762) 2023-07-24 07:55:10,551 DEBUG [RS:1;jenkins-hbase4:46543] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 07:55:10,551 INFO [RS:1;jenkins-hbase4:46543] regionserver.LeaseManager(133): Closed leases 2023-07-24 07:55:10,552 INFO [RS:1;jenkins-hbase4:46543] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-24 07:55:10,552 INFO [RS:1;jenkins-hbase4:46543] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 07:55:10,552 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 07:55:10,552 INFO [RS:1;jenkins-hbase4:46543] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 07:55:10,552 INFO [RS:1;jenkins-hbase4:46543] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 07:55:10,553 INFO [RS:1;jenkins-hbase4:46543] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46543 2023-07-24 07:55:10,558 DEBUG [RS:0;jenkins-hbase4:46877] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/oldWALs 2023-07-24 07:55:10,558 INFO [RS:0;jenkins-hbase4:46877] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46877%2C1690185199045:(num 1690185200762) 2023-07-24 07:55:10,558 DEBUG [RS:0;jenkins-hbase4:46877] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 07:55:10,558 INFO [RS:0;jenkins-hbase4:46877] regionserver.LeaseManager(133): Closed leases 2023-07-24 07:55:10,558 INFO [RS:0;jenkins-hbase4:46877] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-24 07:55:10,559 INFO [RS:0;jenkins-hbase4:46877] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 07:55:10,559 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 07:55:10,559 INFO [RS:0;jenkins-hbase4:46877] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 07:55:10,559 INFO [RS:0;jenkins-hbase4:46877] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 07:55:10,561 INFO [RS:0;jenkins-hbase4:46877] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46877 2023-07-24 07:55:10,561 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 07:55:10,561 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 07:55:10,561 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 07:55:10,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:10,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 07:55:10,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:10,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46543,1690185199091 2023-07-24 07:55:10,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 07:55:10,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 07:55:10,562 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 07:55:10,563 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:10,563 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:10,563 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46877,1690185199045 2023-07-24 07:55:10,563 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46543,1690185199091] 2023-07-24 07:55:10,564 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46543,1690185199091; numProcessing=1 2023-07-24 07:55:10,566 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46543,1690185199091 already deleted, retry=false 2023-07-24 07:55:10,566 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46543,1690185199091 expired; onlineServers=2 2023-07-24 07:55:10,567 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46877,1690185199045] 2023-07-24 07:55:10,567 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46877,1690185199045; numProcessing=2 2023-07-24 07:55:10,568 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46877,1690185199045 already deleted, retry=false 2023-07-24 07:55:10,568 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46877,1690185199045 expired; onlineServers=1 2023-07-24 07:55:10,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/namespace/61dcffa18562b861467de5a803476e2c/.tmp/info/5a4332872f304fefbb43ec1d42bdd371 2023-07-24 07:55:10,579 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/namespace/61dcffa18562b861467de5a803476e2c/.tmp/info/5a4332872f304fefbb43ec1d42bdd371 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/namespace/61dcffa18562b861467de5a803476e2c/info/5a4332872f304fefbb43ec1d42bdd371 2023-07-24 07:55:10,593 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/namespace/61dcffa18562b861467de5a803476e2c/info/5a4332872f304fefbb43ec1d42bdd371, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 07:55:10,594 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 61dcffa18562b861467de5a803476e2c in 58ms, sequenceid=6, compaction requested=false 2023-07-24 07:55:10,599 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/namespace/61dcffa18562b861467de5a803476e2c/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-24 07:55:10,599 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:55:10,600 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:55:10,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 61dcffa18562b861467de5a803476e2c: 2023-07-24 07:55:10,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690185201171.61dcffa18562b861467de5a803476e2c. 2023-07-24 07:55:10,665 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 07:55:10,665 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46543-0x1019651e89e0002, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 07:55:10,665 INFO [RS:1;jenkins-hbase4:46543] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46543,1690185199091; zookeeper connection closed. 2023-07-24 07:55:10,665 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2accf41] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2accf41 2023-07-24 07:55:10,670 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-24 07:55:10,670 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-24 07:55:10,738 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-24 07:55:10,765 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 07:55:10,765 INFO [RS:0;jenkins-hbase4:46877] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46877,1690185199045; zookeeper connection closed. 2023-07-24 07:55:10,765 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:46877-0x1019651e89e0001, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 07:55:10,765 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@100510b6] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@100510b6 2023-07-24 07:55:10,938 DEBUG [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-24 07:55:10,990 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/.tmp/info/d7bbb96d64594348934d5ffcdace15e2 2023-07-24 07:55:10,994 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7bbb96d64594348934d5ffcdace15e2 2023-07-24 07:55:11,007 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/.tmp/rep_barrier/fcd32e0709764d22a1fc7ece5110a36d 2023-07-24 07:55:11,010 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fcd32e0709764d22a1fc7ece5110a36d 2023-07-24 07:55:11,020 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/.tmp/table/15d37c6b62a9407796143502e58d6b8c 2023-07-24 07:55:11,023 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 15d37c6b62a9407796143502e58d6b8c 2023-07-24 07:55:11,023 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/.tmp/info/d7bbb96d64594348934d5ffcdace15e2 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/info/d7bbb96d64594348934d5ffcdace15e2 2023-07-24 07:55:11,027 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d7bbb96d64594348934d5ffcdace15e2 2023-07-24 07:55:11,027 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/info/d7bbb96d64594348934d5ffcdace15e2, entries=59, sequenceid=210, filesize=11.1 K 2023-07-24 07:55:11,028 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/.tmp/rep_barrier/fcd32e0709764d22a1fc7ece5110a36d as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/rep_barrier/fcd32e0709764d22a1fc7ece5110a36d 2023-07-24 07:55:11,031 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fcd32e0709764d22a1fc7ece5110a36d 2023-07-24 07:55:11,031 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/rep_barrier/fcd32e0709764d22a1fc7ece5110a36d, entries=24, sequenceid=210, filesize=7.5 K 2023-07-24 07:55:11,031 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/.tmp/table/15d37c6b62a9407796143502e58d6b8c as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/table/15d37c6b62a9407796143502e58d6b8c 2023-07-24 07:55:11,034 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 15d37c6b62a9407796143502e58d6b8c 2023-07-24 07:55:11,035 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/table/15d37c6b62a9407796143502e58d6b8c, entries=36, sequenceid=210, filesize=8.0 K 2023-07-24 07:55:11,035 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 498ms, sequenceid=210, compaction requested=false 2023-07-24 07:55:11,040 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-07-24 07:55:11,040 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 07:55:11,040 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 07:55:11,041 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-24 07:55:11,041 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-24 07:55:11,041 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-24 07:55:11,138 INFO [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37809,1690185199136; all regions closed. 2023-07-24 07:55:11,145 DEBUG [RS:2;jenkins-hbase4:37809] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/oldWALs 2023-07-24 07:55:11,145 INFO [RS:2;jenkins-hbase4:37809] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37809%2C1690185199136.meta:.meta(num 1690185200885) 2023-07-24 07:55:11,149 DEBUG [RS:2;jenkins-hbase4:37809] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/oldWALs 2023-07-24 07:55:11,149 INFO [RS:2;jenkins-hbase4:37809] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37809%2C1690185199136:(num 1690185200762) 2023-07-24 07:55:11,149 DEBUG [RS:2;jenkins-hbase4:37809] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 07:55:11,149 INFO [RS:2;jenkins-hbase4:37809] regionserver.LeaseManager(133): Closed leases 2023-07-24 07:55:11,150 INFO [RS:2;jenkins-hbase4:37809] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-24 07:55:11,151 INFO [RS:2;jenkins-hbase4:37809] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37809 2023-07-24 07:55:11,151 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 07:55:11,153 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 07:55:11,153 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37809,1690185199136 2023-07-24 07:55:11,155 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37809,1690185199136] 2023-07-24 07:55:11,155 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37809,1690185199136; numProcessing=3 2023-07-24 07:55:11,156 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,37809,1690185199136 already deleted, retry=false 2023-07-24 07:55:11,156 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37809,1690185199136 expired; onlineServers=0 2023-07-24 07:55:11,156 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,37123,1690185197841' ***** 2023-07-24 07:55:11,156 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-24 07:55:11,157 DEBUG [M:0;jenkins-hbase4:37123] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@26560597, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 07:55:11,157 INFO [M:0;jenkins-hbase4:37123] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37123,1690185197841 2023-07-24 07:55:11,157 INFO [M:0;jenkins-hbase4:37123] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37123,1690185197841; all regions closed. 2023-07-24 07:55:11,157 DEBUG [M:0;jenkins-hbase4:37123] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 07:55:11,157 DEBUG [M:0;jenkins-hbase4:37123] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-24 07:55:11,157 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-24 07:55:11,157 DEBUG [M:0;jenkins-hbase4:37123] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-24 07:55:11,157 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690185200350] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690185200350,5,FailOnTimeoutGroup] 2023-07-24 07:55:11,157 INFO [M:0;jenkins-hbase4:37123] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-24 07:55:11,157 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690185200352] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690185200352,5,FailOnTimeoutGroup] 2023-07-24 07:55:11,158 INFO [M:0;jenkins-hbase4:37123] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-24 07:55:11,158 INFO [M:0;jenkins-hbase4:37123] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-24 07:55:11,158 DEBUG [M:0;jenkins-hbase4:37123] master.HMaster(1512): Stopping service threads 2023-07-24 07:55:11,158 INFO [M:0;jenkins-hbase4:37123] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-24 07:55:11,158 ERROR [M:0;jenkins-hbase4:37123] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-07-24 07:55:11,158 INFO [M:0;jenkins-hbase4:37123] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 07:55:11,158 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-24 07:55:11,159 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-24 07:55:11,159 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 07:55:11,159 DEBUG [M:0;jenkins-hbase4:37123] zookeeper.ZKUtil(398): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-24 07:55:11,159 WARN [M:0;jenkins-hbase4:37123] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-24 07:55:11,159 INFO [M:0;jenkins-hbase4:37123] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 07:55:11,159 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 07:55:11,159 INFO [M:0;jenkins-hbase4:37123] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-24 07:55:11,159 DEBUG [M:0;jenkins-hbase4:37123] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-24 07:55:11,159 INFO [M:0;jenkins-hbase4:37123] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 07:55:11,159 DEBUG [M:0;jenkins-hbase4:37123] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 07:55:11,159 DEBUG [M:0;jenkins-hbase4:37123] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-24 07:55:11,159 DEBUG [M:0;jenkins-hbase4:37123] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 07:55:11,159 INFO [M:0;jenkins-hbase4:37123] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.14 KB heapSize=719.10 KB 2023-07-24 07:55:11,181 INFO [M:0;jenkins-hbase4:37123] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.14 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/43bd2012e93c4a72af0aaa5ede7f6238 2023-07-24 07:55:11,185 INFO [M:0;jenkins-hbase4:37123] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 43bd2012e93c4a72af0aaa5ede7f6238 2023-07-24 07:55:11,186 DEBUG [M:0;jenkins-hbase4:37123] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/43bd2012e93c4a72af0aaa5ede7f6238 as hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/43bd2012e93c4a72af0aaa5ede7f6238 2023-07-24 07:55:11,189 INFO [M:0;jenkins-hbase4:37123] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 43bd2012e93c4a72af0aaa5ede7f6238 2023-07-24 07:55:11,189 INFO [M:0;jenkins-hbase4:37123] regionserver.HStore(1080): Added hdfs://localhost:34211/user/jenkins/test-data/7583d3a1-e267-87b9-7bab-9a8be2cbec99/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/43bd2012e93c4a72af0aaa5ede7f6238, entries=156, sequenceid=1282, filesize=34.9 K 2023-07-24 07:55:11,190 INFO [M:0;jenkins-hbase4:37123] regionserver.HRegion(2948): Finished flush of dataSize ~608.14 KB/622736, heapSize ~719.09 KB/736344, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 31ms, sequenceid=1282, compaction requested=false 2023-07-24 07:55:11,191 INFO [M:0;jenkins-hbase4:37123] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 07:55:11,191 DEBUG [M:0;jenkins-hbase4:37123] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 07:55:11,197 INFO [M:0;jenkins-hbase4:37123] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-24 07:55:11,197 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 07:55:11,198 INFO [M:0;jenkins-hbase4:37123] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37123 2023-07-24 07:55:11,199 DEBUG [M:0;jenkins-hbase4:37123] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37123,1690185197841 already deleted, retry=false 2023-07-24 07:55:11,255 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 07:55:11,255 INFO [RS:2;jenkins-hbase4:37809] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37809,1690185199136; zookeeper connection closed. 2023-07-24 07:55:11,256 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): regionserver:37809-0x1019651e89e0003, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 07:55:11,256 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2414450c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2414450c 2023-07-24 07:55:11,256 INFO [Listener at localhost/42537] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-07-24 07:55:11,356 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 07:55:11,356 INFO [M:0;jenkins-hbase4:37123] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37123,1690185197841; zookeeper connection closed. 2023-07-24 07:55:11,356 DEBUG [Listener at localhost/42537-EventThread] zookeeper.ZKWatcher(600): master:37123-0x1019651e89e0000, quorum=127.0.0.1:55594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 07:55:11,358 WARN [Listener at localhost/42537] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-24 07:55:11,362 INFO [Listener at localhost/42537] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 07:55:11,471 WARN [BP-80517894-172.31.14.131-1690185194253 heartbeating to localhost/127.0.0.1:34211] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-24 07:55:11,472 WARN [BP-80517894-172.31.14.131-1690185194253 heartbeating to localhost/127.0.0.1:34211] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-80517894-172.31.14.131-1690185194253 (Datanode Uuid b4241ba3-003c-4dbd-8dcc-7026b912408a) service to localhost/127.0.0.1:34211 2023-07-24 07:55:11,474 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data5/current/BP-80517894-172.31.14.131-1690185194253] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 07:55:11,474 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data6/current/BP-80517894-172.31.14.131-1690185194253] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 07:55:11,476 WARN [Listener at localhost/42537] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-24 07:55:11,479 INFO [Listener at localhost/42537] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 07:55:11,581 WARN [BP-80517894-172.31.14.131-1690185194253 heartbeating to localhost/127.0.0.1:34211] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-80517894-172.31.14.131-1690185194253 (Datanode Uuid 1d2b2de7-56a0-45bb-9a79-d5935e59fefd) service to localhost/127.0.0.1:34211 2023-07-24 07:55:11,582 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data3/current/BP-80517894-172.31.14.131-1690185194253] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 07:55:11,582 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data4/current/BP-80517894-172.31.14.131-1690185194253] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 07:55:11,584 WARN [Listener at localhost/42537] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-24 07:55:11,588 INFO [Listener at localhost/42537] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 07:55:11,613 WARN [BP-80517894-172.31.14.131-1690185194253 heartbeating to localhost/127.0.0.1:34211] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-80517894-172.31.14.131-1690185194253 (Datanode Uuid cda8ebfd-f055-417e-8edc-61fd19134adf) service to localhost/127.0.0.1:34211 2023-07-24 07:55:11,614 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data1/current/BP-80517894-172.31.14.131-1690185194253] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 07:55:11,614 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/b22fba1c-9a14-9199-e48b-b7442643af97/cluster_a8bee194-8ddd-b3a9-2f65-fd28eedb238f/dfs/data/data2/current/BP-80517894-172.31.14.131-1690185194253] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 07:55:11,714 INFO [Listener at localhost/42537] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 07:55:11,908 INFO [Listener at localhost/42537] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-24 07:55:12,154 INFO [Listener at localhost/42537] hbase.HBaseTestingUtility(1293): Minicluster is down