2023-11-01 07:55:26,540 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-01 07:55:26,822 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-01 07:55:26,936 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-01 07:55:27,000 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-01 07:55:27,007 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-01 07:55:27,021 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=171, AvailableMemoryMB=11553
2023-11-01 07:55:27,023 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-01 07:55:27,023 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-01 07:55:27,140 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1035123370:84 , srv1035645750:84 , srv1064567761:84 , srv1121454831:84 , srv1159251581:84 , srv1162834108:84 , srv1232040327:84 , srv1234882061:84 , srv1258446708:84 , srv127149427:84 , srv1335638503:84 , srv1368415916:84 , srv1369674611:84 , srv1390544695:84 , srv1396404457:84 , srv1419477147:84 , srv1431422795:84 , srv1443872582:84 , srv1444811878:84 , srv1506180187:84 , srv1522937251:84 , srv1526861361:84 , srv1532795308:84 , srv160446083:84 , srv1618505681:84 , srv1633826525:84 , srv1634238348:84 , srv1654888555:84 , srv1670998886:84 , srv1671966229:84 , srv1696699104:84 , srv1717928270:84 , srv1741012499:84 , srv1742326578:84 , srv1774145433:84 , srv1797438151:84 , srv1826256619:84 , srv1843090047:84 , srv1858611948:84 , srv186228787:84 , srv1889421657:84 , srv1890133271:84 , srv1895563413:84 , srv1908859718:84 , srv1918968472:84 , srv1932290909:84 , srv1958792637:84 , srv1971160875:84 , srv199108679:84 , srv2018848275:84 , srv2024875925:84 , srv2025714132:84 , srv2026875797:84 , srv2050958270:84 , srv2089138837:84 , srv2111222139:84 , srv2143893674:84 , srv224144578:84 , srv252467128:84 , srv264425720:84 , srv27245713:84 , srv2870408:84 , srv316913752:84 , srv318909907:84 , srv325376917:84 , srv357009647:84 , srv37044338:84 , srv37650185:84 , srv377398305:84 , srv39128116:84 , srv461979643:84 , srv479206429:84 , srv520786062:84 , srv522484887:84 , srv540653263:84 , srv555045626:84 , srv556326597:84 , srv590512863:84 , srv591019739:84 , srv627259909:84 , srv634434738:84 , srv636720669:84 , srv657795269:84 , srv671861161:84 , srv701819948:84 , srv725766588:84 , srv755233597:84 , srv765826764:84 , srv794875961:84 , srv81567918:84 , srv819321195:84 , srv85346071:84 , srv871251418:84 , srv895673488:84 , srv901414837:84 , srv909798276:84 , srv924414713:84 , srv927913959:84 , srv98531154:84 , srv1505100634:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-01 07:55:27,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 07:55:27,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1532795308=23, srv1826256619=37, srv81567918=90, srv1431422795=16, srv794875961=89, srv895673488=94, srv1368415916=11, srv1654888555=28, srv1932290909=46, srv765826764=88, srv1369674611=12, srv1889421657=41, srv590512863=78, srv627259909=80, srv1797438151=36, srv37650185=68, srv591019739=79, srv1443872582=17, srv755233597=87, srv2025714132=52, srv2018848275=50, srv1526861361=22, srv1618505681=25, srv1717928270=32, srv325376917=65, srv556326597=77, srv316913752=63, srv357009647=66, srv725766588=86, srv461979643=71, srv1035123370=0, srv1035645750=1, srv1234882061=7, srv657795269=83, srv318909907=64, srv252467128=59, srv479206429=72, srv1064567761=2, srv2024875925=51, srv199108679=49, srv1159251581=4, srv1506180187=20, srv1895563413=43, srv1396404457=14, srv1121454831=3, srv927913959=98, srv127149427=9, srv1232040327=6, srv1670998886=29, srv1908859718=44, srv1774145433=35, srv2870408=62, srv186228787=40, srv264425720=60, srv1971160875=48, srv224144578=58, srv2143893674=57, srv1843090047=38, srv540653263=75, srv2111222139=56, srv2050958270=54, srv1335638503=10, srv377398305=69, srv1742326578=34, srv1162834108=5, srv2089138837=55, srv701819948=85, srv1444811878=18, srv1390544695=13, srv671861161=84, srv37044338=67, srv2026875797=53, srv27245713=61, srv555045626=76, srv1258446708=8, srv1918968472=45, srv901414837=95, srv1634238348=27, srv924414713=97, srv85346071=92, srv39128116=70, srv1522937251=21, srv634434738=81, srv819321195=91, srv1741012499=33, srv522484887=74, srv98531154=99, srv909798276=96, srv636720669=82, srv1890133271=42, srv520786062=73, srv1696699104=31, srv1505100634=19, srv1958792637=47, srv1419477147=15, srv871251418=93, srv1633826525=26, srv1858611948=39, srv160446083=24, srv1671966229=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 07:55:27,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 07:55:27,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 07:55:27,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 07:55:27,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 07:55:27,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 07:55:27,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 07:55:27,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 07:55:27,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 07:55:27,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 07:55:27,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 07:55:27,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 07:55:27,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 07:55:27,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 07:55:27,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 07:55:27,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 07:55:27,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 07:55:27,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 07:55:27,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 07:55:27,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 07:55:27,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 07:55:27,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 07:55:27,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 07:55:27,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 07:55:27,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 07:55:27,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 07:55:27,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 07:55:27,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 07:55:27,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 07:55:27,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 07:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 07:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-01 07:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-01 07:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-01 07:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-01 07:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-01 07:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-01 07:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-01 07:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-01 07:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-01 07:55:27,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-01 07:55:27,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-01 07:55:27,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-01 07:55:27,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-01 07:55:27,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-01 07:55:27,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-01 07:55:27,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-01 07:55:27,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-01 07:55:27,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-01 07:55:27,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-01 07:55:27,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-01 07:55:27,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-01 07:55:27,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-01 07:55:27,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-01 07:55:27,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-01 07:55:27,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-01 07:55:27,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-01 07:55:27,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-01 07:55:27,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-01 07:55:27,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-01 07:55:27,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-01 07:55:27,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-01 07:55:27,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-01 07:55:27,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-01 07:55:27,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-01 07:55:27,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-01 07:55:27,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-01 07:55:27,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-01 07:55:27,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-01 07:55:27,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-01 07:55:27,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-01 07:55:27,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-01 07:55:27,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-01 07:55:27,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-01 07:55:27,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-01 07:55:27,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-01 07:55:27,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-01 07:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-01 07:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-01 07:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-01 07:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-01 07:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-01 07:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-01 07:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-01 07:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-01 07:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-01 07:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-01 07:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-01 07:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-01 07:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-01 07:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-01 07:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-01 07:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-01 07:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-01 07:55:27,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-01 07:55:27,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-01 07:55:27,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-01 07:55:27,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-01 07:55:27,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-01 07:55:27,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-01 07:55:27,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-01 07:55:27,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-01 07:55:27,455 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:55:27,455 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-01 07:55:27,456 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-01 07:55:27,861 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 587 ms to try 5000 different iterations.  Found a solution that moves 4684 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18302588568841216. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040160235237433256, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0530082686305625, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5204444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040160235237433256, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17833333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:55:27,882 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1532795308:82 , srv1064567761:83 , srv1232040327:83 , srv1431422795:83 , srv1634238348:83 , srv316913752:83 , srv794875961:83 , srv1234882061:84 , srv1258446708:84 , srv1396404457:84 , srv1443872582:84 , srv1671966229:84 , srv1742326578:84 , srv1774145433:84 , srv1908859718:84 , srv199108679:84 , srv318909907:84 , srv37044338:84 , srv377398305:84 , srv591019739:84 , srv627259909:84 , srv636720669:84 , srv671861161:84 , srv81567918:84 , srv1035645750:85 , srv1335638503:85 , srv160446083:85 , srv1797438151:85 , srv1889421657:85 , srv1918968472:85 , srv1932290909:85 , srv2018848275:85 , srv2089138837:85 , srv2111222139:85 , srv252467128:85 , srv264425720:85 , srv357009647:85 , srv522484887:85 , srv540653263:85 , srv634434738:85 , srv85346071:85 , srv909798276:85 , srv1368415916:86 , srv1419477147:86 , srv1506180187:86 , srv1526861361:86 , srv1654888555:86 , srv1826256619:86 , srv1958792637:86 , srv27245713:86 , srv37650185:86 , srv479206429:86 , srv657795269:86 , srv765826764:86 , srv901414837:86 , srv127149427:87 , srv1369674611:87 , srv1390544695:87 , srv1444811878:87 , srv1633826525:87 , srv1670998886:87 , srv1890133271:87 , srv1971160875:87 , srv325376917:87 , srv520786062:87 , srv701819948:87 , srv725766588:87 , srv755233597:87 , srv871251418:87 , srv895673488:87 , srv1035123370:88 , srv1121454831:88 , srv1162834108:88 , srv1522937251:88 , srv1618505681:88 , srv1741012499:88 , srv1843090047:88 , srv1895563413:88 , srv2024875925:88 , srv2870408:88 , srv556326597:88 , srv927913959:88 , srv1159251581:89 , srv1717928270:89 , srv186228787:89 , srv2025714132:89 , srv2143893674:89 , srv461979643:89 , srv555045626:89 , srv819321195:89 , srv924414713:89 , srv2026875797:90 , srv2050958270:90 , srv590512863:90 , srv1696699104:92 , srv1858611948:92 , srv224144578:92 , srv39128116:92 , srv98531154:97 , srv1505100634:447 }
2023-11-01 07:55:27,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 07:55:27,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1532795308=23, srv1826256619=37, srv81567918=90, srv1431422795=16, srv794875961=89, srv895673488=94, srv1368415916=11, srv1654888555=28, srv1932290909=46, srv765826764=88, srv1369674611=12, srv1889421657=41, srv590512863=78, srv627259909=80, srv1797438151=36, srv37650185=68, srv591019739=79, srv1443872582=17, srv755233597=87, srv2025714132=52, srv2018848275=50, srv1526861361=22, srv1618505681=25, srv1717928270=32, srv325376917=65, srv556326597=77, srv316913752=63, srv357009647=66, srv725766588=86, srv461979643=71, srv1035123370=0, srv1035645750=1, srv1234882061=7, srv657795269=83, srv318909907=64, srv252467128=59, srv479206429=72, srv1064567761=2, srv2024875925=51, srv199108679=49, srv1159251581=4, srv1506180187=20, srv1895563413=43, srv1396404457=14, srv1121454831=3, srv927913959=98, srv127149427=9, srv1232040327=6, srv1670998886=29, srv1908859718=44, srv1774145433=35, srv2870408=62, srv186228787=40, srv264425720=60, srv1971160875=48, srv224144578=58, srv2143893674=57, srv1843090047=38, srv540653263=75, srv2111222139=56, srv2050958270=54, srv1335638503=10, srv377398305=69, srv1742326578=34, srv1162834108=5, srv2089138837=55, srv701819948=85, srv1444811878=18, srv1390544695=13, srv671861161=84, srv37044338=67, srv2026875797=53, srv27245713=61, srv555045626=76, srv1258446708=8, srv1918968472=45, srv901414837=95, srv1634238348=27, srv924414713=97, srv85346071=92, srv39128116=70, srv1522937251=21, srv634434738=81, srv819321195=91, srv1741012499=33, srv522484887=74, srv98531154=99, srv909798276=96, srv636720669=82, srv1890133271=42, srv520786062=73, srv1696699104=31, srv1505100634=19, srv1958792637=47, srv1419477147=15, srv871251418=93, srv1633826525=26, srv1858611948=39, srv160446083=24, srv1671966229=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 07:55:27,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 07:55:27,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 07:55:27,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 07:55:27,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-01 07:55:27,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-01 07:55:27,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-01 07:55:27,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-01 07:55:27,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-01 07:55:27,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-01 07:55:27,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-01 07:55:27,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-01 07:55:27,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-01 07:55:27,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-01 07:55:27,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-01 07:55:28,027 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040160235237433256, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0530082686305625, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040160235237433256, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17833333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:55:28,027 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-01 07:55:28,027 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18299308761940494, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040160235237433256, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0530082686305625, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040160235237433256, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17833333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-01 07:55:28,308 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 382 ms to try 5000 different iterations.  Found a solution that moves 2629 regions; Going from a computed imbalance of 0.18299308761940494 to a new imbalance of 0.017054644861355422. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023058344320405006); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03499559055116361, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2921111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023058344320405006); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04708333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:55:28,319 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv98531154:76 , srv1858611948:78 , srv39128116:79 , srv2050958270:81 , srv224144578:81 , srv461979643:81 , srv819321195:81 , srv1696699104:82 , srv1843090047:82 , srv2026875797:82 , srv927913959:82 , srv1159251581:83 , srv2024875925:83 , srv2143893674:83 , srv555045626:83 , srv895673488:83 , srv1121454831:84 , srv1162834108:84 , srv1444811878:84 , srv1633826525:84 , srv1717928270:84 , srv186228787:84 , srv2025714132:84 , srv2870408:84 , srv590512863:84 , srv701819948:84 , srv755233597:84 , srv1035123370:85 , srv127149427:85 , srv1368415916:85 , srv1369674611:85 , srv1390544695:85 , srv1419477147:85 , srv1506180187:85 , srv1522937251:85 , srv1618505681:85 , srv1654888555:85 , srv1670998886:85 , srv1741012499:85 , srv1826256619:85 , srv1958792637:85 , srv479206429:85 , srv520786062:85 , srv556326597:85 , srv657795269:85 , srv901414837:85 , srv924414713:85 , srv1232040327:86 , srv1234882061:86 , srv1742326578:86 , srv1797438151:86 , srv1918968472:86 , srv1932290909:86 , srv1971160875:86 , srv2018848275:86 , srv2111222139:86 , srv252467128:86 , srv27245713:86 , srv318909907:86 , srv37650185:86 , srv377398305:86 , srv634434738:86 , srv81567918:86 , srv871251418:86 , srv1035645750:87 , srv1890133271:87 , srv1895563413:87 , srv1908859718:87 , srv357009647:87 , srv37044338:87 , srv522484887:87 , srv540653263:87 , srv725766588:87 , srv765826764:87 , srv909798276:87 , srv1064567761:88 , srv1335638503:88 , srv1396404457:88 , srv1431422795:88 , srv1443872582:88 , srv1532795308:88 , srv1671966229:88 , srv1774145433:88 , srv1889421657:88 , srv2089138837:88 , srv264425720:88 , srv591019739:88 , srv627259909:88 , srv636720669:88 , srv85346071:88 , srv1258446708:89 , srv1526861361:89 , srv160446083:89 , srv199108679:89 , srv316913752:89 , srv671861161:89 , srv794875961:89 , srv1634238348:91 , srv325376917:91 , srv1505100634:532 }
2023-11-01 07:55:28,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 07:55:28,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1532795308=23, srv1826256619=37, srv81567918=90, srv1431422795=16, srv794875961=89, srv895673488=94, srv1368415916=11, srv1654888555=28, srv1932290909=46, srv765826764=88, srv1369674611=12, srv1889421657=41, srv590512863=78, srv627259909=80, srv1797438151=36, srv37650185=68, srv591019739=79, srv1443872582=17, srv755233597=87, srv2025714132=52, srv2018848275=50, srv1526861361=22, srv1618505681=25, srv1717928270=32, srv325376917=65, srv556326597=77, srv316913752=63, srv357009647=66, srv725766588=86, srv461979643=71, srv1035123370=0, srv1035645750=1, srv1234882061=7, srv657795269=83, srv318909907=64, srv252467128=59, srv479206429=72, srv1064567761=2, srv2024875925=51, srv199108679=49, srv1159251581=4, srv1506180187=20, srv1895563413=43, srv1396404457=14, srv1121454831=3, srv927913959=98, srv127149427=9, srv1232040327=6, srv1670998886=29, srv1908859718=44, srv1774145433=35, srv2870408=62, srv186228787=40, srv264425720=60, srv1971160875=48, srv224144578=58, srv2143893674=57, srv1843090047=38, srv540653263=75, srv2111222139=56, srv2050958270=54, srv1335638503=10, srv377398305=69, srv1742326578=34, srv1162834108=5, srv2089138837=55, srv701819948=85, srv1444811878=18, srv1390544695=13, srv671861161=84, srv37044338=67, srv2026875797=53, srv27245713=61, srv555045626=76, srv1258446708=8, srv1918968472=45, srv901414837=95, srv1634238348=27, srv924414713=97, srv85346071=92, srv39128116=70, srv1522937251=21, srv634434738=81, srv819321195=91, srv1741012499=33, srv522484887=74, srv98531154=99, srv909798276=96, srv636720669=82, srv1890133271=42, srv520786062=73, srv1696699104=31, srv1505100634=19, srv1958792637=47, srv1419477147=15, srv871251418=93, srv1633826525=26, srv1858611948=39, srv160446083=24, srv1671966229=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 07:55:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 07:55:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 07:55:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 07:55:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 07:55:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 07:55:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 07:55:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 07:55:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 07:55:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 07:55:28,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 07:55:28,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-01 07:55:28,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-01 07:55:28,348 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-01 07:55:28,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-01 07:55:28,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-01 07:55:28,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-01 07:55:28,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-01 07:55:28,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-01 07:55:28,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-01 07:55:28,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-01 07:55:28,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-01 07:55:28,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-01 07:55:28,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-01 07:55:28,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-01 07:55:28,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-01 07:55:28,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-01 07:55:28,469 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023058344320405006); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03499559055116361, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023058344320405006); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04708333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:55:28,470 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-01 07:55:28,470 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.017036236209899424, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023058344320405006); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03499559055116361, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023058344320405006); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04708333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-01 07:55:28,734 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 367 ms to try 5000 different iterations.  Found a solution that moves 1069 regions; Going from a computed imbalance of 0.017036236209899424 to a new imbalance of 6.792477385100904E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006559178363224968); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00721639236343101); MoveCostFunction : (multiplier=7.0, imbalance=0.11877777777777777, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006559178363224968); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00425); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:55:28,736 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv325376917:79 , srv725766588:82 , srv1526861361:83 , srv1671966229:83 , srv1890133271:83 , srv924414713:83 , srv1035123370:84 , srv1522937251:84 , srv160446083:84 , srv1634238348:84 , srv1696699104:84 , srv1741012499:84 , srv186228787:84 , srv1889421657:84 , srv1895563413:84 , srv2089138837:84 , srv224144578:84 , srv357009647:84 , srv522484887:84 , srv556326597:84 , srv590512863:84 , srv636720669:84 , srv871251418:84 , srv1035645750:85 , srv1159251581:85 , srv1258446708:85 , srv127149427:85 , srv1335638503:85 , srv1369674611:85 , srv1390544695:85 , srv1396404457:85 , srv1618505681:85 , srv1717928270:85 , srv1774145433:85 , srv1971160875:85 , srv199108679:85 , srv2025714132:85 , srv2143893674:85 , srv264425720:85 , srv27245713:85 , srv520786062:85 , srv591019739:85 , srv671861161:85 , srv765826764:85 , srv909798276:85 , srv98531154:85 , srv1121454831:86 , srv1162834108:86 , srv1368415916:86 , srv1431422795:86 , srv1443872582:86 , srv1444811878:86 , srv1633826525:86 , srv1670998886:86 , srv1797438151:86 , srv1858611948:86 , srv1918968472:86 , srv1958792637:86 , srv2018848275:86 , srv2024875925:86 , srv2026875797:86 , srv2050958270:86 , srv252467128:86 , srv2870408:86 , srv316913752:86 , srv37650185:86 , srv540653263:86 , srv627259909:86 , srv701819948:86 , srv794875961:86 , srv81567918:86 , srv85346071:86 , srv901414837:86 , srv1064567761:87 , srv1232040327:87 , srv1234882061:87 , srv1419477147:87 , srv1506180187:87 , srv1532795308:87 , srv1654888555:87 , srv1742326578:87 , srv1826256619:87 , srv1843090047:87 , srv1908859718:87 , srv1932290909:87 , srv2111222139:87 , srv318909907:87 , srv37044338:87 , srv377398305:87 , srv39128116:87 , srv461979643:87 , srv555045626:87 , srv634434738:87 , srv657795269:87 , srv755233597:87 , srv895673488:87 , srv479206429:88 , srv819321195:88 , srv927913959:88 , srv1505100634:537 }
2023-11-01 07:55:28,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 07:55:28,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1532795308=23, srv1826256619=37, srv81567918=90, srv1431422795=16, srv794875961=89, srv895673488=94, srv1368415916=11, srv1654888555=28, srv1932290909=46, srv765826764=88, srv1369674611=12, srv1889421657=41, srv590512863=78, srv627259909=80, srv1797438151=36, srv37650185=68, srv591019739=79, srv1443872582=17, srv755233597=87, srv2025714132=52, srv2018848275=50, srv1526861361=22, srv1618505681=25, srv1717928270=32, srv325376917=65, srv556326597=77, srv316913752=63, srv357009647=66, srv725766588=86, srv461979643=71, srv1035123370=0, srv1035645750=1, srv1234882061=7, srv657795269=83, srv318909907=64, srv252467128=59, srv479206429=72, srv1064567761=2, srv2024875925=51, srv199108679=49, srv1159251581=4, srv1506180187=20, srv1895563413=43, srv1396404457=14, srv1121454831=3, srv927913959=98, srv127149427=9, srv1232040327=6, srv1670998886=29, srv1908859718=44, srv1774145433=35, srv2870408=62, srv186228787=40, srv264425720=60, srv1971160875=48, srv224144578=58, srv2143893674=57, srv1843090047=38, srv540653263=75, srv2111222139=56, srv2050958270=54, srv1335638503=10, srv377398305=69, srv1742326578=34, srv1162834108=5, srv2089138837=55, srv701819948=85, srv1444811878=18, srv1390544695=13, srv671861161=84, srv37044338=67, srv2026875797=53, srv27245713=61, srv555045626=76, srv1258446708=8, srv1918968472=45, srv901414837=95, srv1634238348=27, srv924414713=97, srv85346071=92, srv39128116=70, srv1522937251=21, srv634434738=81, srv819321195=91, srv1741012499=33, srv522484887=74, srv98531154=99, srv909798276=96, srv636720669=82, srv1890133271=42, srv520786062=73, srv1696699104=31, srv1505100634=19, srv1958792637=47, srv1419477147=15, srv871251418=93, srv1633826525=26, srv1858611948=39, srv160446083=24, srv1671966229=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 07:55:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 07:55:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 07:55:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 07:55:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 07:55:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 07:55:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 07:55:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 07:55:28,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 07:55:28,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 07:55:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 07:55:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 07:55:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 07:55:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 07:55:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 07:55:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 07:55:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-01 07:55:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-01 07:55:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-01 07:55:28,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-01 07:55:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-01 07:55:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-01 07:55:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-01 07:55:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-01 07:55:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-01 07:55:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-01 07:55:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-01 07:55:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-01 07:55:28,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-01 07:55:28,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-01 07:55:28,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-01 07:55:28,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-01 07:55:28,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-01 07:55:28,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-01 07:55:28,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-01 07:55:28,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-01 07:55:28,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-01 07:55:28,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-01 07:55:28,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-01 07:55:28,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-01 07:55:28,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-01 07:55:28,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-01 07:55:28,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-01 07:55:28,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-01 07:55:28,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-01 07:55:28,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-01 07:55:28,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-01 07:55:28,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-01 07:55:28,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-01 07:55:28,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-01 07:55:28,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-01 07:55:28,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-01 07:55:28,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-01 07:55:28,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-01 07:55:28,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-01 07:55:28,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-01 07:55:28,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-01 07:55:28,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-01 07:55:28,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-01 07:55:28,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-01 07:55:28,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-01 07:55:28,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-01 07:55:28,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-01 07:55:28,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-01 07:55:28,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-01 07:55:28,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-01 07:55:28,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-01 07:55:28,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-01 07:55:28,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-01 07:55:28,865 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006559178363224968); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00721639236343101); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006559178363224968); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00425); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:55:28,865 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-01 07:55:28,866 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.717624405236085E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006559178363224968); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00721639236343101); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006559178363224968); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00425); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-01 07:55:29,235 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 463 ms to try 5000 different iterations.  Found a solution that moves 287 regions; Going from a computed imbalance of 6.717624405236085E-4 to a new imbalance of 2.4766344221487485E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0028818873975125464); MoveCostFunction : (multiplier=7.0, imbalance=0.03188888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:55:29,236 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1121454831:83 , srv1419477147:83 , srv1618505681:83 , srv1895563413:83 , srv479206429:83 , srv555045626:83 , srv590512863:83 , srv85346071:83 , srv98531154:83 , srv1064567761:84 , srv1162834108:84 , srv1258446708:84 , srv1335638503:84 , srv1443872582:84 , srv1506180187:84 , srv1634238348:84 , srv1670998886:84 , srv1696699104:84 , srv1717928270:84 , srv1826256619:84 , srv1908859718:84 , srv1932290909:84 , srv1971160875:84 , srv199108679:84 , srv2024875925:84 , srv2025714132:84 , srv2111222139:84 , srv264425720:84 , srv2870408:84 , srv316913752:84 , srv37044338:84 , srv39128116:84 , srv461979643:84 , srv540653263:84 , srv591019739:84 , srv627259909:84 , srv634434738:84 , srv657795269:84 , srv671861161:84 , srv755233597:84 , srv765826764:84 , srv794875961:84 , srv819321195:84 , srv901414837:84 , srv927913959:84 , srv1035645750:85 , srv1234882061:85 , srv127149427:85 , srv1368415916:85 , srv1369674611:85 , srv1390544695:85 , srv1396404457:85 , srv1431422795:85 , srv1444811878:85 , srv1522937251:85 , srv1526861361:85 , srv1532795308:85 , srv160446083:85 , srv1633826525:85 , srv1654888555:85 , srv1741012499:85 , srv1742326578:85 , srv1774145433:85 , srv1797438151:85 , srv1843090047:85 , srv186228787:85 , srv1889421657:85 , srv1918968472:85 , srv1958792637:85 , srv2018848275:85 , srv2026875797:85 , srv2050958270:85 , srv2089138837:85 , srv2143893674:85 , srv224144578:85 , srv252467128:85 , srv27245713:85 , srv318909907:85 , srv325376917:85 , srv37650185:85 , srv377398305:85 , srv556326597:85 , srv701819948:85 , srv871251418:85 , srv895673488:85 , srv909798276:85 , srv924414713:85 , srv1035123370:86 , srv1159251581:86 , srv1232040327:86 , srv1858611948:86 , srv1890133271:86 , srv357009647:86 , srv520786062:86 , srv522484887:86 , srv636720669:86 , srv725766588:86 , srv81567918:86 , srv1671966229:87 , srv1505100634:626 }
2023-11-01 07:55:29,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 07:55:29,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1532795308=23, srv1826256619=37, srv81567918=90, srv1431422795=16, srv794875961=89, srv895673488=94, srv1368415916=11, srv1654888555=28, srv1932290909=46, srv765826764=88, srv1369674611=12, srv1889421657=41, srv590512863=78, srv627259909=80, srv1797438151=36, srv37650185=68, srv591019739=79, srv1443872582=17, srv755233597=87, srv2025714132=52, srv2018848275=50, srv1526861361=22, srv1618505681=25, srv1717928270=32, srv325376917=65, srv556326597=77, srv316913752=63, srv357009647=66, srv725766588=86, srv461979643=71, srv1035123370=0, srv1035645750=1, srv1234882061=7, srv657795269=83, srv318909907=64, srv252467128=59, srv479206429=72, srv1064567761=2, srv2024875925=51, srv199108679=49, srv1159251581=4, srv1506180187=20, srv1895563413=43, srv1396404457=14, srv1121454831=3, srv927913959=98, srv127149427=9, srv1232040327=6, srv1670998886=29, srv1908859718=44, srv1774145433=35, srv2870408=62, srv186228787=40, srv264425720=60, srv1971160875=48, srv224144578=58, srv2143893674=57, srv1843090047=38, srv540653263=75, srv2111222139=56, srv2050958270=54, srv1335638503=10, srv377398305=69, srv1742326578=34, srv1162834108=5, srv2089138837=55, srv701819948=85, srv1444811878=18, srv1390544695=13, srv671861161=84, srv37044338=67, srv2026875797=53, srv27245713=61, srv555045626=76, srv1258446708=8, srv1918968472=45, srv901414837=95, srv1634238348=27, srv924414713=97, srv85346071=92, srv39128116=70, srv1522937251=21, srv634434738=81, srv819321195=91, srv1741012499=33, srv522484887=74, srv98531154=99, srv909798276=96, srv636720669=82, srv1890133271=42, srv520786062=73, srv1696699104=31, srv1505100634=19, srv1958792637=47, srv1419477147=15, srv871251418=93, srv1633826525=26, srv1858611948=39, srv160446083=24, srv1671966229=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 07:55:29,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 07:55:29,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-01 07:55:29,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-01 07:55:29,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-01 07:55:29,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-01 07:55:29,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-01 07:55:29,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-01 07:55:29,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-01 07:55:29,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-01 07:55:29,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-01 07:55:29,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-01 07:55:29,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-01 07:55:29,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-01 07:55:29,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-01 07:55:29,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-01 07:55:29,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-01 07:55:29,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-01 07:55:29,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-01 07:55:29,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-01 07:55:29,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-01 07:55:29,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-01 07:55:29,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-01 07:55:29,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-01 07:55:29,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-01 07:55:29,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-01 07:55:29,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-01 07:55:29,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-01 07:55:29,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-01 07:55:29,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-01 07:55:29,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-01 07:55:29,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-01 07:55:29,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-01 07:55:29,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-01 07:55:29,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-01 07:55:29,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-01 07:55:29,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-01 07:55:29,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-01 07:55:29,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-01 07:55:29,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-01 07:55:29,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-01 07:55:29,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-01 07:55:29,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-01 07:55:29,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-01 07:55:29,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-01 07:55:29,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-01 07:55:29,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-01 07:55:29,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-01 07:55:29,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-01 07:55:29,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-01 07:55:29,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-01 07:55:29,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-01 07:55:29,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-01 07:55:29,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-01 07:55:29,361 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=2.2756727269083098E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0028818873975125464); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:55:29,362 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1121454831:83 , srv1419477147:83 , srv1618505681:83 , srv1895563413:83 , srv479206429:83 , srv555045626:83 , srv590512863:83 , srv85346071:83 , srv98531154:83 , srv1064567761:84 , srv1162834108:84 , srv1258446708:84 , srv1335638503:84 , srv1443872582:84 , srv1506180187:84 , srv1634238348:84 , srv1670998886:84 , srv1696699104:84 , srv1717928270:84 , srv1826256619:84 , srv1908859718:84 , srv1932290909:84 , srv1971160875:84 , srv199108679:84 , srv2024875925:84 , srv2025714132:84 , srv2111222139:84 , srv264425720:84 , srv2870408:84 , srv316913752:84 , srv37044338:84 , srv39128116:84 , srv461979643:84 , srv540653263:84 , srv591019739:84 , srv627259909:84 , srv634434738:84 , srv657795269:84 , srv671861161:84 , srv755233597:84 , srv765826764:84 , srv794875961:84 , srv819321195:84 , srv901414837:84 , srv927913959:84 , srv1035645750:85 , srv1234882061:85 , srv127149427:85 , srv1368415916:85 , srv1369674611:85 , srv1390544695:85 , srv1396404457:85 , srv1431422795:85 , srv1444811878:85 , srv1522937251:85 , srv1526861361:85 , srv1532795308:85 , srv160446083:85 , srv1633826525:85 , srv1654888555:85 , srv1741012499:85 , srv1742326578:85 , srv1774145433:85 , srv1797438151:85 , srv1843090047:85 , srv186228787:85 , srv1889421657:85 , srv1918968472:85 , srv1958792637:85 , srv2018848275:85 , srv2026875797:85 , srv2050958270:85 , srv2089138837:85 , srv2143893674:85 , srv224144578:85 , srv252467128:85 , srv27245713:85 , srv318909907:85 , srv325376917:85 , srv37650185:85 , srv377398305:85 , srv556326597:85 , srv701819948:85 , srv871251418:85 , srv895673488:85 , srv909798276:85 , srv924414713:85 , srv1035123370:86 , srv1159251581:86 , srv1232040327:86 , srv1858611948:86 , srv1890133271:86 , srv357009647:86 , srv520786062:86 , srv522484887:86 , srv636720669:86 , srv725766588:86 , srv81567918:86 , srv1671966229:87 , srv1505100634:626 }
2023-11-01 07:55:29,389 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=231 (was 231), ProcessCount=171 (was 171), AvailableMemoryMB=11052 (was 11553)
2023-11-01 07:55:29,399 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=171, AvailableMemoryMB=11052
2023-11-01 07:55:29,400 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-01 07:55:29,400 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-01 07:55:29,408 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1032274584:84 , srv1047996707:84 , srv1056559458:84 , srv1057574684:84 , srv1142436490:84 , srv1142573356:84 , srv1174050191:84 , srv1239025520:84 , srv1347473104:84 , srv1431529327:84 , srv1491668206:84 , srv1601724371:84 , srv1702263958:84 , srv1771505513:84 , srv1775934769:84 , srv1838902423:84 , srv1874334833:84 , srv1918566022:84 , srv1982652739:84 , srv2003119799:84 , srv293965601:84 , srv401971452:84 , srv409669677:84 , srv489504446:84 , srv558336390:84 , srv697418679:84 , srv758918483:84 , srv900669483:84 , srv934577543:84 , srv2006488652:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-01 07:55:29,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 07:55:29,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1775934769=14, srv1874334833=16, srv1918566022=17, srv293965601=21, srv1174050191=6, srv697418679=26, srv1142436490=4, srv1601724371=11, srv1982652739=18, srv1771505513=13, srv2003119799=19, srv2006488652=20, srv1702263958=12, srv1347473104=8, srv900669483=28, srv558336390=25, srv489504446=24, srv1431529327=9, srv401971452=22, srv758918483=27, srv934577543=29, srv1239025520=7, srv1057574684=3, srv1142573356=5, srv1491668206=10, srv409669677=23, srv1047996707=1, srv1838902423=15, srv1032274584=0, srv1056559458=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 07:55:29,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 07:55:29,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-01 07:55:29,436 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.9970349778429514, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:55:29,436 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912183700576283, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.9970349778429514, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-01 07:57:29,423 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5403565 different iterations.  Found a solution that moves 1855 regions; Going from a computed imbalance of 0.9912183700576283 to a new imbalance of 4.329662539733032E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.687037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:57:29,431 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1032274584:90 , srv1047996707:90 , srv1056559458:90 , srv1057574684:90 , srv1142436490:90 , srv1142573356:90 , srv1174050191:90 , srv1239025520:90 , srv1347473104:90 , srv1431529327:90 , srv1491668206:90 , srv1601724371:90 , srv1702263958:90 , srv1771505513:90 , srv1775934769:90 , srv1838902423:90 , srv1874334833:90 , srv1918566022:90 , srv1982652739:90 , srv2003119799:90 , srv2006488652:90 , srv293965601:90 , srv401971452:90 , srv409669677:90 , srv489504446:90 , srv558336390:90 , srv697418679:90 , srv758918483:90 , srv900669483:90 , srv934577543:90 }
2023-11-01 07:57:29,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 07:57:29,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1775934769=14, srv1874334833=16, srv1918566022=17, srv293965601=21, srv1174050191=6, srv697418679=26, srv1142436490=4, srv1601724371=11, srv1982652739=18, srv1771505513=13, srv2003119799=19, srv2006488652=20, srv1702263958=12, srv1347473104=8, srv900669483=28, srv558336390=25, srv489504446=24, srv1431529327=9, srv401971452=22, srv758918483=27, srv934577543=29, srv1239025520=7, srv1057574684=3, srv1142573356=5, srv1491668206=10, srv409669677=23, srv1047996707=1, srv1838902423=15, srv1032274584=0, srv1056559458=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 07:57:29,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 07:57:29,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 07:57:29,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 07:57:29,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 07:57:29,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 07:57:29,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-01 07:57:29,452 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 07:57:29,452 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1032274584:90 , srv1047996707:90 , srv1056559458:90 , srv1057574684:90 , srv1142436490:90 , srv1142573356:90 , srv1174050191:90 , srv1239025520:90 , srv1347473104:90 , srv1431529327:90 , srv1491668206:90 , srv1601724371:90 , srv1702263958:90 , srv1771505513:90 , srv1775934769:90 , srv1838902423:90 , srv1874334833:90 , srv1918566022:90 , srv1982652739:90 , srv2003119799:90 , srv2006488652:90 , srv293965601:90 , srv401971452:90 , srv409669677:90 , srv489504446:90 , srv558336390:90 , srv697418679:90 , srv758918483:90 , srv900669483:90 , srv934577543:90 }
2023-11-01 07:57:29,466 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=179 (was 231), ProcessCount=170 (was 171), AvailableMemoryMB=11001 (was 11052)