2023-05-28 23:53:24,480 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-28 23:53:24,759 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-28 23:53:24,869 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-28 23:53:24,923 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-28 23:53:24,930 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-28 23:53:24,942 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=172, AvailableMemoryMB=5044
2023-05-28 23:53:24,944 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-28 23:53:24,945 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-28 23:53:25,077 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1005775087:84 , srv101269514:84 , srv1059780936:84 , srv1073543273:84 , srv1097332446:84 , srv1119379818:84 , srv1175672516:84 , srv1201568987:84 , srv1234659820:84 , srv1237171199:84 , srv1249319105:84 , srv13314895:84 , srv134409581:84 , srv1374407054:84 , srv1398374210:84 , srv1403206437:84 , srv1497177295:84 , srv1571473778:84 , srv1584284669:84 , srv1599751452:84 , srv1627427228:84 , srv1642664851:84 , srv1662163865:84 , srv1663044064:84 , srv16702881:84 , srv169253727:84 , srv1726880144:84 , srv1748918647:84 , srv1758114718:84 , srv1785721460:84 , srv1805881160:84 , srv1842710531:84 , srv1855949527:84 , srv1861567584:84 , srv1871209086:84 , srv1872414916:84 , srv1874492479:84 , srv189575441:84 , srv1898931400:84 , srv1917782781:84 , srv1967950815:84 , srv1972024243:84 , srv2009555460:84 , srv2056177540:84 , srv2066457405:84 , srv2069246035:84 , srv2069703912:84 , srv2076896296:84 , srv2078944595:84 , srv2138029169:84 , srv228071866:84 , srv250631325:84 , srv259599742:84 , srv274130237:84 , srv292033535:84 , srv292217250:84 , srv296166699:84 , srv297033659:84 , srv314083004:84 , srv329333470:84 , srv331696844:84 , srv414280891:84 , srv421178339:84 , srv429136233:84 , srv449090977:84 , srv455258521:84 , srv458188272:84 , srv466691363:84 , srv478615449:84 , srv492504021:84 , srv532883483:84 , srv562880562:84 , srv574688413:84 , srv579446670:84 , srv602509354:84 , srv631841860:84 , srv635931793:84 , srv638902960:84 , srv652553740:84 , srv661005417:84 , srv680875593:84 , srv702191164:84 , srv730492025:84 , srv732159432:84 , srv736397349:84 , srv746587544:84 , srv757936973:84 , srv805499888:84 , srv823041984:84 , srv829847235:84 , srv838992318:84 , srv850364920:84 , srv864973648:84 , srv885088047:84 , srv906272125:84 , srv959023567:84 , srv993584333:84 , srv994568681:84 , srv99611103:84 , srv1181765016:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-28 23:53:25,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-28 23:53:25,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1398374210=15, srv746587544=86, srv2078944595=49, srv259599742=53, srv1571473778=18, srv1855949527=33, srv1917782781=40, srv838992318=91, srv99611103=99, srv1073543273=3, srv2076896296=48, srv1181765016=7, srv1627427228=21, srv1872414916=36, srv2138029169=50, srv1861567584=34, srv250631325=52, srv2056177540=44, srv329333470=60, srv959023567=96, srv574688413=73, srv1249319105=11, srv314083004=59, srv1967950815=41, srv1758114718=29, srv1584284669=19, srv631841860=76, srv635931793=77, srv1175672516=6, srv16702881=25, srv429136233=64, srv1234659820=9, srv2069246035=46, srv757936973=87, srv458188272=67, srv13314895=12, srv1097332446=4, srv680875593=81, srv532883483=71, srv1119379818=5, srv579446670=74, srv1005775087=0, srv455258521=66, srv1805881160=31, srv1403206437=16, srv652553740=79, srv1599751452=20, srv2066457405=45, srv562880562=72, srv1726880144=27, srv2009555460=43, srv1663044064=24, srv331696844=61, srv702191164=82, srv101269514=1, srv1842710531=32, srv805499888=88, srv829847235=90, srv885088047=94, srv1374407054=14, srv449090977=65, srv1874492479=37, srv1662163865=23, srv993584333=97, srv1871209086=35, srv414280891=62, srv1898931400=39, srv1059780936=2, srv466691363=68, srv2069703912=47, srv994568681=98, srv228071866=51, srv732159432=84, srv1748918647=28, srv169253727=26, srv661005417=80, srv864973648=93, srv602509354=75, srv638902960=78, srv850364920=92, srv1972024243=42, srv296166699=57, srv274130237=54, srv297033659=58, srv478615449=69, srv1785721460=30, srv421178339=63, srv906272125=95, srv492504021=70, srv1201568987=8, srv823041984=89, srv736397349=85, srv1642664851=22, srv730492025=83, srv189575441=38, srv1497177295=17, srv292217250=56, srv292033535=55, srv1237171199=10, srv134409581=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-28 23:53:25,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-28 23:53:25,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-28 23:53:25,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-28 23:53:25,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-28 23:53:25,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-28 23:53:25,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-28 23:53:25,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-28 23:53:25,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-28 23:53:25,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-28 23:53:25,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-28 23:53:25,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-28 23:53:25,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-28 23:53:25,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-28 23:53:25,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-28 23:53:25,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-28 23:53:25,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-28 23:53:25,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-28 23:53:25,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-28 23:53:25,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-28 23:53:25,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-28 23:53:25,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-28 23:53:25,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-28 23:53:25,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-28 23:53:25,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-28 23:53:25,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-28 23:53:25,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-28 23:53:25,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-28 23:53:25,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-28 23:53:25,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-28 23:53:25,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-28 23:53:25,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-28 23:53:25,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-28 23:53:25,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-28 23:53:25,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-28 23:53:25,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-28 23:53:25,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-28 23:53:25,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-28 23:53:25,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-28 23:53:25,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-28 23:53:25,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-28 23:53:25,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-28 23:53:25,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-28 23:53:25,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-28 23:53:25,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-28 23:53:25,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-28 23:53:25,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-28 23:53:25,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-28 23:53:25,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-28 23:53:25,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-28 23:53:25,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-28 23:53:25,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-28 23:53:25,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-28 23:53:25,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-28 23:53:25,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-28 23:53:25,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-28 23:53:25,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-28 23:53:25,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-28 23:53:25,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-28 23:53:25,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-28 23:53:25,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-28 23:53:25,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-28 23:53:25,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-28 23:53:25,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-28 23:53:25,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-28 23:53:25,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-28 23:53:25,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-28 23:53:25,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-28 23:53:25,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-28 23:53:25,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-28 23:53:25,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-28 23:53:25,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-28 23:53:25,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-28 23:53:25,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-28 23:53:25,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-28 23:53:25,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-28 23:53:25,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-28 23:53:25,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-28 23:53:25,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-28 23:53:25,317 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:25,318 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-28 23:53:25,318 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-28 23:53:25,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 399 ms to try 5000 different iterations.  Found a solution that moves 4690 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17964710108339188. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04181533522086569, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05895079650462529, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5211111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04181533522086569, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17491666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:25,601 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv455258521:81 , srv1571473778:82 , srv2078944595:82 , srv274130237:82 , srv746587544:82 , srv101269514:83 , srv1748918647:83 , srv1842710531:83 , srv2066457405:83 , srv1758114718:84 , srv1872414916:84 , srv2069246035:84 , srv635931793:84 , srv661005417:84 , srv680875593:84 , srv823041984:84 , srv838992318:84 , srv1201568987:85 , srv1398374210:85 , srv1497177295:85 , srv1599751452:85 , srv1642664851:85 , srv1861567584:85 , srv1871209086:85 , srv1967950815:85 , srv2009555460:85 , srv2056177540:85 , srv2076896296:85 , srv250631325:85 , srv297033659:85 , srv466691363:85 , srv730492025:85 , srv732159432:85 , srv736397349:85 , srv805499888:85 , srv829847235:85 , srv864973648:85 , srv885088047:85 , srv993584333:85 , srv1005775087:86 , srv1059780936:86 , srv1374407054:86 , srv1584284669:86 , srv1662163865:86 , srv1663044064:86 , srv1874492479:86 , srv189575441:86 , srv1917782781:86 , srv1972024243:86 , srv2069703912:86 , srv2138029169:86 , srv574688413:86 , srv702191164:86 , srv757936973:86 , srv959023567:86 , srv99611103:86 , srv1073543273:87 , srv1175672516:87 , srv1249319105:87 , srv1627427228:87 , srv1855949527:87 , srv259599742:87 , srv329333470:87 , srv331696844:87 , srv429136233:87 , srv602509354:87 , srv994568681:87 , srv1119379818:88 , srv1234659820:88 , srv13314895:88 , srv16702881:88 , srv169253727:88 , srv1726880144:88 , srv1785721460:88 , srv228071866:88 , srv296166699:88 , srv314083004:88 , srv421178339:88 , srv449090977:88 , srv532883483:88 , srv562880562:88 , srv579446670:88 , srv631841860:88 , srv652553740:88 , srv850364920:88 , srv906272125:88 , srv1097332446:89 , srv1237171199:89 , srv1403206437:89 , srv1805881160:89 , srv292033535:89 , srv292217250:89 , srv414280891:89 , srv638902960:89 , srv134409581:90 , srv1898931400:90 , srv478615449:90 , srv492504021:90 , srv458188272:92 , srv1181765016:462 }
2023-05-28 23:53:25,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-28 23:53:25,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1398374210=15, srv746587544=86, srv2078944595=49, srv259599742=53, srv1571473778=18, srv1855949527=33, srv1917782781=40, srv838992318=91, srv99611103=99, srv1073543273=3, srv2076896296=48, srv1181765016=7, srv1627427228=21, srv1872414916=36, srv2138029169=50, srv1861567584=34, srv250631325=52, srv2056177540=44, srv329333470=60, srv959023567=96, srv574688413=73, srv1249319105=11, srv314083004=59, srv1967950815=41, srv1758114718=29, srv1584284669=19, srv631841860=76, srv635931793=77, srv1175672516=6, srv16702881=25, srv429136233=64, srv1234659820=9, srv2069246035=46, srv757936973=87, srv458188272=67, srv13314895=12, srv1097332446=4, srv680875593=81, srv532883483=71, srv1119379818=5, srv579446670=74, srv1005775087=0, srv455258521=66, srv1805881160=31, srv1403206437=16, srv652553740=79, srv1599751452=20, srv2066457405=45, srv562880562=72, srv1726880144=27, srv2009555460=43, srv1663044064=24, srv331696844=61, srv702191164=82, srv101269514=1, srv1842710531=32, srv805499888=88, srv829847235=90, srv885088047=94, srv1374407054=14, srv449090977=65, srv1874492479=37, srv1662163865=23, srv993584333=97, srv1871209086=35, srv414280891=62, srv1898931400=39, srv1059780936=2, srv466691363=68, srv2069703912=47, srv994568681=98, srv228071866=51, srv732159432=84, srv1748918647=28, srv169253727=26, srv661005417=80, srv864973648=93, srv602509354=75, srv638902960=78, srv850364920=92, srv1972024243=42, srv296166699=57, srv274130237=54, srv297033659=58, srv478615449=69, srv1785721460=30, srv421178339=63, srv906272125=95, srv492504021=70, srv1201568987=8, srv823041984=89, srv736397349=85, srv1642664851=22, srv730492025=83, srv189575441=38, srv1497177295=17, srv292217250=56, srv292033535=55, srv1237171199=10, srv134409581=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-28 23:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-28 23:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-28 23:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-28 23:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-28 23:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-28 23:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-28 23:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-28 23:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-28 23:53:25,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-28 23:53:25,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-28 23:53:25,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-28 23:53:25,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-28 23:53:25,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-28 23:53:25,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-28 23:53:25,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-28 23:53:25,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-28 23:53:25,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-28 23:53:25,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-28 23:53:25,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-28 23:53:25,740 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04181533522086569, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05895079650462529, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04181533522086569, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17491666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:25,740 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-28 23:53:25,740 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17961426100148675, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04181533522086569, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05895079650462529, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04181533522086569, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17491666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-28 23:53:26,032 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 393 ms to try 5000 different iterations.  Found a solution that moves 2631 regions; Going from a computed imbalance of 0.17961426100148675 to a new imbalance of 0.014621096574016626. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02260831820098064); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03318485449570684, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29233333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02260831820098064); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044333333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:26,047 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv458188272:78 , srv134409581:80 , srv292033535:81 , srv1237171199:82 , srv13314895:82 , srv1403206437:82 , srv292217250:82 , srv532883483:82 , srv562880562:82 , srv631841860:82 , srv638902960:82 , srv1073543273:83 , srv1234659820:83 , srv1726880144:83 , srv1805881160:83 , srv414280891:83 , srv421178339:83 , srv478615449:83 , srv652553740:83 , srv1097332446:84 , srv1119379818:84 , srv16702881:84 , srv169253727:84 , srv1874492479:84 , srv1898931400:84 , srv1917782781:84 , srv429136233:84 , srv449090977:84 , srv492504021:84 , srv906272125:84 , srv1005775087:85 , srv1201568987:85 , srv1497177295:85 , srv1599751452:85 , srv1785721460:85 , srv1967950815:85 , srv1972024243:85 , srv296166699:85 , srv1175672516:86 , srv1374407054:86 , srv1584284669:86 , srv1627427228:86 , srv1642664851:86 , srv189575441:86 , srv2076896296:86 , srv2138029169:86 , srv228071866:86 , srv250631325:86 , srv259599742:86 , srv331696844:86 , srv574688413:86 , srv579446670:86 , srv661005417:86 , srv732159432:86 , srv736397349:86 , srv757936973:86 , srv805499888:86 , srv850364920:86 , srv959023567:86 , srv994568681:86 , srv1059780936:87 , srv1249319105:87 , srv1662163865:87 , srv1663044064:87 , srv1842710531:87 , srv1855949527:87 , srv1871209086:87 , srv2009555460:87 , srv2069246035:87 , srv2069703912:87 , srv297033659:87 , srv602509354:87 , srv702191164:87 , srv829847235:87 , srv838992318:87 , srv864973648:87 , srv885088047:87 , srv99611103:87 , srv1398374210:88 , srv1758114718:88 , srv1861567584:88 , srv1872414916:88 , srv2056177540:88 , srv314083004:88 , srv329333470:88 , srv466691363:88 , srv635931793:88 , srv823041984:88 , srv1571473778:89 , srv1748918647:89 , srv2078944595:89 , srv680875593:89 , srv730492025:89 , srv746587544:89 , srv101269514:90 , srv2066457405:90 , srv993584333:90 , srv274130237:91 , srv455258521:91 , srv1181765016:513 }
2023-05-28 23:53:26,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-28 23:53:26,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1398374210=15, srv746587544=86, srv2078944595=49, srv259599742=53, srv1571473778=18, srv1855949527=33, srv1917782781=40, srv838992318=91, srv99611103=99, srv1073543273=3, srv2076896296=48, srv1181765016=7, srv1627427228=21, srv1872414916=36, srv2138029169=50, srv1861567584=34, srv250631325=52, srv2056177540=44, srv329333470=60, srv959023567=96, srv574688413=73, srv1249319105=11, srv314083004=59, srv1967950815=41, srv1758114718=29, srv1584284669=19, srv631841860=76, srv635931793=77, srv1175672516=6, srv16702881=25, srv429136233=64, srv1234659820=9, srv2069246035=46, srv757936973=87, srv458188272=67, srv13314895=12, srv1097332446=4, srv680875593=81, srv532883483=71, srv1119379818=5, srv579446670=74, srv1005775087=0, srv455258521=66, srv1805881160=31, srv1403206437=16, srv652553740=79, srv1599751452=20, srv2066457405=45, srv562880562=72, srv1726880144=27, srv2009555460=43, srv1663044064=24, srv331696844=61, srv702191164=82, srv101269514=1, srv1842710531=32, srv805499888=88, srv829847235=90, srv885088047=94, srv1374407054=14, srv449090977=65, srv1874492479=37, srv1662163865=23, srv993584333=97, srv1871209086=35, srv414280891=62, srv1898931400=39, srv1059780936=2, srv466691363=68, srv2069703912=47, srv994568681=98, srv228071866=51, srv732159432=84, srv1748918647=28, srv169253727=26, srv661005417=80, srv864973648=93, srv602509354=75, srv638902960=78, srv850364920=92, srv1972024243=42, srv296166699=57, srv274130237=54, srv297033659=58, srv478615449=69, srv1785721460=30, srv421178339=63, srv906272125=95, srv492504021=70, srv1201568987=8, srv823041984=89, srv736397349=85, srv1642664851=22, srv730492025=83, srv189575441=38, srv1497177295=17, srv292217250=56, srv292033535=55, srv1237171199=10, srv134409581=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-28 23:53:26,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-28 23:53:26,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-28 23:53:26,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-28 23:53:26,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-28 23:53:26,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-28 23:53:26,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-28 23:53:26,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-28 23:53:26,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-28 23:53:26,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-28 23:53:26,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-28 23:53:26,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-28 23:53:26,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-28 23:53:26,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-28 23:53:26,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-28 23:53:26,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-28 23:53:26,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-28 23:53:26,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-28 23:53:26,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-28 23:53:26,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-28 23:53:26,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-28 23:53:26,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-28 23:53:26,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-28 23:53:26,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-28 23:53:26,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-28 23:53:26,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-28 23:53:26,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-28 23:53:26,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-28 23:53:26,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-28 23:53:26,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-28 23:53:26,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-28 23:53:26,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-28 23:53:26,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-28 23:53:26,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-28 23:53:26,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-28 23:53:26,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-28 23:53:26,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-28 23:53:26,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-28 23:53:26,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-28 23:53:26,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-28 23:53:26,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-28 23:53:26,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-28 23:53:26,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-28 23:53:26,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-28 23:53:26,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-28 23:53:26,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-28 23:53:26,249 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02260831820098064); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03318485449570684, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02260831820098064); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044333333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:26,249 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-28 23:53:26,249 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01460267391826131, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02260831820098064); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03318485449570684, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02260831820098064); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044333333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-28 23:53:26,585 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 478 ms to try 5000 different iterations.  Found a solution that moves 975 regions; Going from a computed imbalance of 0.01460267391826131 to a new imbalance of 6.611155621832217E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007454396416772877); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010709929101971124); MoveCostFunction : (multiplier=7.0, imbalance=0.10833333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007454396416772877); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:26,587 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv314083004:82 , srv994568681:82 , srv1175672516:83 , srv329333470:83 , srv492504021:83 , srv602509354:83 , srv730492025:83 , srv993584333:83 , srv1059780936:84 , srv1119379818:84 , srv1249319105:84 , srv1374407054:84 , srv1398374210:84 , srv1662163865:84 , srv1663044064:84 , srv1805881160:84 , srv1861567584:84 , srv1898931400:84 , srv2066457405:84 , srv2069703912:84 , srv228071866:84 , srv259599742:84 , srv296166699:84 , srv331696844:84 , srv478615449:84 , srv579446670:84 , srv850364920:84 , srv864973648:84 , srv99611103:84 , srv101269514:85 , srv1097332446:85 , srv1237171199:85 , srv1584284669:85 , srv1627427228:85 , srv1748918647:85 , srv1785721460:85 , srv1855949527:85 , srv1871209086:85 , srv189575441:85 , srv2009555460:85 , srv2056177540:85 , srv2138029169:85 , srv297033659:85 , srv449090977:85 , srv466691363:85 , srv574688413:85 , srv638902960:85 , srv680875593:85 , srv702191164:85 , srv823041984:85 , srv829847235:85 , srv959023567:85 , srv1201568987:86 , srv1642664851:86 , srv16702881:86 , srv169253727:86 , srv1726880144:86 , srv1758114718:86 , srv1972024243:86 , srv2069246035:86 , srv2078944595:86 , srv274130237:86 , srv292217250:86 , srv414280891:86 , srv421178339:86 , srv455258521:86 , srv532883483:86 , srv635931793:86 , srv652553740:86 , srv736397349:86 , srv746587544:86 , srv757936973:86 , srv805499888:86 , srv838992318:86 , srv885088047:86 , srv906272125:86 , srv1234659820:87 , srv13314895:87 , srv1403206437:87 , srv1497177295:87 , srv1571473778:87 , srv1599751452:87 , srv1842710531:87 , srv1872414916:87 , srv1874492479:87 , srv1917782781:87 , srv2076896296:87 , srv250631325:87 , srv292033535:87 , srv429136233:87 , srv458188272:87 , srv562880562:87 , srv631841860:87 , srv661005417:87 , srv732159432:87 , srv1005775087:88 , srv134409581:88 , srv1967950815:88 , srv1073543273:89 , srv1181765016:549 }
2023-05-28 23:53:26,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-28 23:53:26,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1398374210=15, srv746587544=86, srv2078944595=49, srv259599742=53, srv1571473778=18, srv1855949527=33, srv1917782781=40, srv838992318=91, srv99611103=99, srv1073543273=3, srv2076896296=48, srv1181765016=7, srv1627427228=21, srv1872414916=36, srv2138029169=50, srv1861567584=34, srv250631325=52, srv2056177540=44, srv329333470=60, srv959023567=96, srv574688413=73, srv1249319105=11, srv314083004=59, srv1967950815=41, srv1758114718=29, srv1584284669=19, srv631841860=76, srv635931793=77, srv1175672516=6, srv16702881=25, srv429136233=64, srv1234659820=9, srv2069246035=46, srv757936973=87, srv458188272=67, srv13314895=12, srv1097332446=4, srv680875593=81, srv532883483=71, srv1119379818=5, srv579446670=74, srv1005775087=0, srv455258521=66, srv1805881160=31, srv1403206437=16, srv652553740=79, srv1599751452=20, srv2066457405=45, srv562880562=72, srv1726880144=27, srv2009555460=43, srv1663044064=24, srv331696844=61, srv702191164=82, srv101269514=1, srv1842710531=32, srv805499888=88, srv829847235=90, srv885088047=94, srv1374407054=14, srv449090977=65, srv1874492479=37, srv1662163865=23, srv993584333=97, srv1871209086=35, srv414280891=62, srv1898931400=39, srv1059780936=2, srv466691363=68, srv2069703912=47, srv994568681=98, srv228071866=51, srv732159432=84, srv1748918647=28, srv169253727=26, srv661005417=80, srv864973648=93, srv602509354=75, srv638902960=78, srv850364920=92, srv1972024243=42, srv296166699=57, srv274130237=54, srv297033659=58, srv478615449=69, srv1785721460=30, srv421178339=63, srv906272125=95, srv492504021=70, srv1201568987=8, srv823041984=89, srv736397349=85, srv1642664851=22, srv730492025=83, srv189575441=38, srv1497177295=17, srv292217250=56, srv292033535=55, srv1237171199=10, srv134409581=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-28 23:53:26,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-28 23:53:26,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-28 23:53:26,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-28 23:53:26,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-28 23:53:26,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-28 23:53:26,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-28 23:53:26,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-28 23:53:26,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-28 23:53:26,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-28 23:53:26,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-28 23:53:26,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-28 23:53:26,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-28 23:53:26,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-28 23:53:26,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-28 23:53:26,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-28 23:53:26,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-28 23:53:26,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-28 23:53:26,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-28 23:53:26,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-28 23:53:26,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-28 23:53:26,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-28 23:53:26,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-28 23:53:26,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-28 23:53:26,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-28 23:53:26,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-28 23:53:26,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-28 23:53:26,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-28 23:53:26,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-28 23:53:26,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-28 23:53:26,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-28 23:53:26,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-28 23:53:26,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-28 23:53:26,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-28 23:53:26,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-28 23:53:26,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-28 23:53:26,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-28 23:53:26,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-28 23:53:26,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-28 23:53:26,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-28 23:53:26,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-28 23:53:26,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-28 23:53:26,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-28 23:53:26,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-28 23:53:26,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-28 23:53:26,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-28 23:53:26,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-28 23:53:26,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-28 23:53:26,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-28 23:53:26,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-28 23:53:26,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-28 23:53:26,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-28 23:53:26,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-28 23:53:26,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-28 23:53:26,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-28 23:53:26,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-28 23:53:26,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-28 23:53:26,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-28 23:53:26,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-28 23:53:26,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-28 23:53:26,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-28 23:53:26,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-28 23:53:26,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-28 23:53:26,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-28 23:53:26,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-28 23:53:26,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-28 23:53:26,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-28 23:53:26,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-28 23:53:26,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-28 23:53:26,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-28 23:53:26,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-28 23:53:26,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-28 23:53:26,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-28 23:53:26,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-28 23:53:26,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-28 23:53:26,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-28 23:53:26,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-28 23:53:26,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-28 23:53:26,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-28 23:53:26,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-28 23:53:26,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-28 23:53:26,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-28 23:53:26,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-28 23:53:26,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-28 23:53:26,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-28 23:53:26,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-28 23:53:26,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-28 23:53:26,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-28 23:53:26,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-28 23:53:26,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-28 23:53:26,718 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007454396416772877); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010709929101971124); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007454396416772877); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:26,718 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-28 23:53:26,718 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.542884662647748E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007454396416772877); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010709929101971124); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007454396416772877); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-28 23:53:27,100 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 477 ms to try 5000 different iterations.  Found a solution that moves 280 regions; Going from a computed imbalance of 6.542884662647748E-4 to a new imbalance of 3.489949376851439E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.03111111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:27,101 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1073543273:82 , srv1249319105:83 , srv134409581:83 , srv1855949527:83 , srv1872414916:83 , srv757936973:83 , srv1005775087:84 , srv101269514:84 , srv1234659820:84 , srv1398374210:84 , srv1403206437:84 , srv1571473778:84 , srv1627427228:84 , srv16702881:84 , srv169253727:84 , srv1726880144:84 , srv1758114718:84 , srv1785721460:84 , srv1898931400:84 , srv1967950815:84 , srv2009555460:84 , srv2056177540:84 , srv2069703912:84 , srv228071866:84 , srv250631325:84 , srv259599742:84 , srv274130237:84 , srv296166699:84 , srv297033659:84 , srv314083004:84 , srv329333470:84 , srv414280891:84 , srv455258521:84 , srv466691363:84 , srv579446670:84 , srv635931793:84 , srv680875593:84 , srv702191164:84 , srv805499888:84 , srv885088047:84 , srv906272125:84 , srv1059780936:85 , srv1097332446:85 , srv13314895:85 , srv1497177295:85 , srv1584284669:85 , srv1599751452:85 , srv1642664851:85 , srv1662163865:85 , srv1663044064:85 , srv1748918647:85 , srv1842710531:85 , srv1861567584:85 , srv1871209086:85 , srv1874492479:85 , srv189575441:85 , srv1917782781:85 , srv1972024243:85 , srv2066457405:85 , srv2069246035:85 , srv2076896296:85 , srv2078944595:85 , srv2138029169:85 , srv292033535:85 , srv292217250:85 , srv421178339:85 , srv429136233:85 , srv458188272:85 , srv562880562:85 , srv574688413:85 , srv602509354:85 , srv652553740:85 , srv661005417:85 , srv730492025:85 , srv732159432:85 , srv736397349:85 , srv746587544:85 , srv823041984:85 , srv829847235:85 , srv838992318:85 , srv850364920:85 , srv959023567:85 , srv993584333:85 , srv99611103:85 , srv1119379818:86 , srv1175672516:86 , srv1201568987:86 , srv1237171199:86 , srv1374407054:86 , srv1805881160:86 , srv331696844:86 , srv449090977:86 , srv478615449:86 , srv492504021:86 , srv532883483:86 , srv631841860:86 , srv638902960:86 , srv864973648:86 , srv994568681:86 , srv1181765016:618 }
2023-05-28 23:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-28 23:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1398374210=15, srv746587544=86, srv2078944595=49, srv259599742=53, srv1571473778=18, srv1855949527=33, srv1917782781=40, srv838992318=91, srv99611103=99, srv1073543273=3, srv2076896296=48, srv1181765016=7, srv1627427228=21, srv1872414916=36, srv2138029169=50, srv1861567584=34, srv250631325=52, srv2056177540=44, srv329333470=60, srv959023567=96, srv574688413=73, srv1249319105=11, srv314083004=59, srv1967950815=41, srv1758114718=29, srv1584284669=19, srv631841860=76, srv635931793=77, srv1175672516=6, srv16702881=25, srv429136233=64, srv1234659820=9, srv2069246035=46, srv757936973=87, srv458188272=67, srv13314895=12, srv1097332446=4, srv680875593=81, srv532883483=71, srv1119379818=5, srv579446670=74, srv1005775087=0, srv455258521=66, srv1805881160=31, srv1403206437=16, srv652553740=79, srv1599751452=20, srv2066457405=45, srv562880562=72, srv1726880144=27, srv2009555460=43, srv1663044064=24, srv331696844=61, srv702191164=82, srv101269514=1, srv1842710531=32, srv805499888=88, srv829847235=90, srv885088047=94, srv1374407054=14, srv449090977=65, srv1874492479=37, srv1662163865=23, srv993584333=97, srv1871209086=35, srv414280891=62, srv1898931400=39, srv1059780936=2, srv466691363=68, srv2069703912=47, srv994568681=98, srv228071866=51, srv732159432=84, srv1748918647=28, srv169253727=26, srv661005417=80, srv864973648=93, srv602509354=75, srv638902960=78, srv850364920=92, srv1972024243=42, srv296166699=57, srv274130237=54, srv297033659=58, srv478615449=69, srv1785721460=30, srv421178339=63, srv906272125=95, srv492504021=70, srv1201568987=8, srv823041984=89, srv736397349=85, srv1642664851=22, srv730492025=83, srv189575441=38, srv1497177295=17, srv292217250=56, srv292033535=55, srv1237171199=10, srv134409581=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-28 23:53:27,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-28 23:53:27,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-28 23:53:27,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-28 23:53:27,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-28 23:53:27,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-28 23:53:27,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-28 23:53:27,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-28 23:53:27,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-28 23:53:27,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-28 23:53:27,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-28 23:53:27,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-28 23:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-28 23:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-28 23:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-28 23:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-28 23:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-28 23:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-28 23:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-28 23:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-28 23:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-28 23:53:27,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-28 23:53:27,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-28 23:53:27,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-28 23:53:27,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-28 23:53:27,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-28 23:53:27,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-28 23:53:27,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-28 23:53:27,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-28 23:53:27,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-28 23:53:27,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-28 23:53:27,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-28 23:53:27,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-28 23:53:27,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-28 23:53:27,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-28 23:53:27,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-28 23:53:27,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-28 23:53:27,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-28 23:53:27,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-28 23:53:27,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-28 23:53:27,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-28 23:53:27,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-28 23:53:27,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-28 23:53:27,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-28 23:53:27,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-28 23:53:27,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-28 23:53:27,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-28 23:53:27,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-28 23:53:27,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-28 23:53:27,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-28 23:53:27,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-28 23:53:27,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-28 23:53:27,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-28 23:53:27,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-28 23:53:27,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-28 23:53:27,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-28 23:53:27,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-28 23:53:27,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-28 23:53:27,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-28 23:53:27,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-28 23:53:27,235 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:27,235 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-28 23:53:27,235 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.293889186372961E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-28 23:53:27,481 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 346 ms to try 5000 different iterations.  Found a solution that moves 27 regions; Going from a computed imbalance of 3.293889186372961E-5 to a new imbalance of 6.587043713088231E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.003); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:27,481 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1005775087:83 , srv1097332446:83 , srv2076896296:83 , srv274130237:83 , srv331696844:83 , srv429136233:83 , srv449090977:83 , srv478615449:83 , srv492504021:83 , srv631841860:83 , srv732159432:83 , srv850364920:83 , srv993584333:83 , srv101269514:84 , srv1059780936:84 , srv1119379818:84 , srv1175672516:84 , srv1201568987:84 , srv1234659820:84 , srv1237171199:84 , srv13314895:84 , srv1374407054:84 , srv1403206437:84 , srv1497177295:84 , srv1571473778:84 , srv1584284669:84 , srv1599751452:84 , srv1627427228:84 , srv1642664851:84 , srv1662163865:84 , srv1663044064:84 , srv16702881:84 , srv169253727:84 , srv1748918647:84 , srv1758114718:84 , srv1785721460:84 , srv1805881160:84 , srv1842710531:84 , srv1855949527:84 , srv1861567584:84 , srv1871209086:84 , srv1872414916:84 , srv1874492479:84 , srv189575441:84 , srv1898931400:84 , srv1917782781:84 , srv1967950815:84 , srv1972024243:84 , srv2056177540:84 , srv2066457405:84 , srv2069246035:84 , srv2078944595:84 , srv2138029169:84 , srv228071866:84 , srv250631325:84 , srv292033535:84 , srv292217250:84 , srv314083004:84 , srv329333470:84 , srv414280891:84 , srv421178339:84 , srv455258521:84 , srv458188272:84 , srv466691363:84 , srv532883483:84 , srv562880562:84 , srv574688413:84 , srv579446670:84 , srv602509354:84 , srv635931793:84 , srv638902960:84 , srv652553740:84 , srv661005417:84 , srv680875593:84 , srv702191164:84 , srv730492025:84 , srv736397349:84 , srv746587544:84 , srv823041984:84 , srv829847235:84 , srv838992318:84 , srv864973648:84 , srv885088047:84 , srv906272125:84 , srv959023567:84 , srv99611103:84 , srv1073543273:85 , srv1249319105:85 , srv134409581:85 , srv1398374210:85 , srv1726880144:85 , srv2009555460:85 , srv2069703912:85 , srv259599742:85 , srv296166699:85 , srv297033659:85 , srv757936973:85 , srv805499888:85 , srv994568681:85 , srv1181765016:684 }
2023-05-28 23:53:27,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-28 23:53:27,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1398374210=15, srv746587544=86, srv2078944595=49, srv259599742=53, srv1571473778=18, srv1855949527=33, srv1917782781=40, srv838992318=91, srv99611103=99, srv1073543273=3, srv2076896296=48, srv1181765016=7, srv1627427228=21, srv1872414916=36, srv2138029169=50, srv1861567584=34, srv250631325=52, srv2056177540=44, srv329333470=60, srv959023567=96, srv574688413=73, srv1249319105=11, srv314083004=59, srv1967950815=41, srv1758114718=29, srv1584284669=19, srv631841860=76, srv635931793=77, srv1175672516=6, srv16702881=25, srv429136233=64, srv1234659820=9, srv2069246035=46, srv757936973=87, srv458188272=67, srv13314895=12, srv1097332446=4, srv680875593=81, srv532883483=71, srv1119379818=5, srv579446670=74, srv1005775087=0, srv455258521=66, srv1805881160=31, srv1403206437=16, srv652553740=79, srv1599751452=20, srv2066457405=45, srv562880562=72, srv1726880144=27, srv2009555460=43, srv1663044064=24, srv331696844=61, srv702191164=82, srv101269514=1, srv1842710531=32, srv805499888=88, srv829847235=90, srv885088047=94, srv1374407054=14, srv449090977=65, srv1874492479=37, srv1662163865=23, srv993584333=97, srv1871209086=35, srv414280891=62, srv1898931400=39, srv1059780936=2, srv466691363=68, srv2069703912=47, srv994568681=98, srv228071866=51, srv732159432=84, srv1748918647=28, srv169253727=26, srv661005417=80, srv864973648=93, srv602509354=75, srv638902960=78, srv850364920=92, srv1972024243=42, srv296166699=57, srv274130237=54, srv297033659=58, srv478615449=69, srv1785721460=30, srv421178339=63, srv906272125=95, srv492504021=70, srv1201568987=8, srv823041984=89, srv736397349=85, srv1642664851=22, srv730492025=83, srv189575441=38, srv1497177295=17, srv292217250=56, srv292033535=55, srv1237171199=10, srv134409581=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-28 23:53:27,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-28 23:53:27,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-28 23:53:27,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-28 23:53:27,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-28 23:53:27,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-28 23:53:27,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-28 23:53:27,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-28 23:53:27,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-28 23:53:27,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-28 23:53:27,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-28 23:53:27,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-28 23:53:27,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-28 23:53:27,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-28 23:53:27,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-28 23:53:27,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-28 23:53:27,608 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:27,609 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1005775087:83 , srv1097332446:83 , srv2076896296:83 , srv274130237:83 , srv331696844:83 , srv429136233:83 , srv449090977:83 , srv478615449:83 , srv492504021:83 , srv631841860:83 , srv732159432:83 , srv850364920:83 , srv993584333:83 , srv101269514:84 , srv1059780936:84 , srv1119379818:84 , srv1175672516:84 , srv1201568987:84 , srv1234659820:84 , srv1237171199:84 , srv13314895:84 , srv1374407054:84 , srv1403206437:84 , srv1497177295:84 , srv1571473778:84 , srv1584284669:84 , srv1599751452:84 , srv1627427228:84 , srv1642664851:84 , srv1662163865:84 , srv1663044064:84 , srv16702881:84 , srv169253727:84 , srv1748918647:84 , srv1758114718:84 , srv1785721460:84 , srv1805881160:84 , srv1842710531:84 , srv1855949527:84 , srv1861567584:84 , srv1871209086:84 , srv1872414916:84 , srv1874492479:84 , srv189575441:84 , srv1898931400:84 , srv1917782781:84 , srv1967950815:84 , srv1972024243:84 , srv2056177540:84 , srv2066457405:84 , srv2069246035:84 , srv2078944595:84 , srv2138029169:84 , srv228071866:84 , srv250631325:84 , srv292033535:84 , srv292217250:84 , srv314083004:84 , srv329333470:84 , srv414280891:84 , srv421178339:84 , srv455258521:84 , srv458188272:84 , srv466691363:84 , srv532883483:84 , srv562880562:84 , srv574688413:84 , srv579446670:84 , srv602509354:84 , srv635931793:84 , srv638902960:84 , srv652553740:84 , srv661005417:84 , srv680875593:84 , srv702191164:84 , srv730492025:84 , srv736397349:84 , srv746587544:84 , srv823041984:84 , srv829847235:84 , srv838992318:84 , srv864973648:84 , srv885088047:84 , srv906272125:84 , srv959023567:84 , srv99611103:84 , srv1073543273:85 , srv1249319105:85 , srv134409581:85 , srv1398374210:85 , srv1726880144:85 , srv2009555460:85 , srv2069703912:85 , srv259599742:85 , srv296166699:85 , srv297033659:85 , srv757936973:85 , srv805499888:85 , srv994568681:85 , srv1181765016:684 }
2023-05-28 23:53:27,631 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=271 (was 271), ProcessCount=172 (was 172), AvailableMemoryMB=4538 (was 5044)
2023-05-28 23:53:27,638 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=172, AvailableMemoryMB=4538
2023-05-28 23:53:27,639 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-28 23:53:27,639 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-28 23:53:27,647 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1110390962:84 , srv1152188336:84 , srv1153847676:84 , srv1181468868:84 , srv1251151497:84 , srv1253969260:84 , srv1333309623:84 , srv1392973777:84 , srv1415177715:84 , srv1464985163:84 , srv1528523691:84 , srv1581583791:84 , srv1581996072:84 , srv176334624:84 , srv1783407394:84 , srv1816195644:84 , srv1922901752:84 , srv1948761277:84 , srv262563110:84 , srv29515365:84 , srv394145958:84 , srv403365334:84 , srv610343380:84 , srv640380137:84 , srv660159197:84 , srv691183524:84 , srv751818423:84 , srv857671118:84 , srv881690727:84 , srv1293105913:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-28 23:53:27,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-28 23:53:27,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv29515365=20, srv881690727=29, srv1251151497=4, srv1253969260=5, srv1392973777=8, srv1528523691=11, srv751818423=27, srv660159197=25, srv1152188336=1, srv403365334=22, srv610343380=23, srv857671118=28, srv1464985163=10, srv1816195644=16, srv262563110=19, srv1153847676=2, srv1333309623=7, srv1581996072=13, srv1922901752=17, srv176334624=14, srv640380137=24, srv1110390962=0, srv691183524=26, srv1415177715=9, srv1948761277=18, srv1783407394=15, srv1181468868=3, srv394145958=21, srv1293105913=6, srv1581583791=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-28 23:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-28 23:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-28 23:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-28 23:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-28 23:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-28 23:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-28 23:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-28 23:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-28 23:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-28 23:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-28 23:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-28 23:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-28 23:53:27,673 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.112495948417553, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:53:27,673 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912547514324414, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.112495948417553, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-28 23:55:27,660 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6061126 different iterations.  Found a solution that moves 1854 regions; Going from a computed imbalance of 0.9912547514324414 to a new imbalance of 4.3273284898463826E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6866666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:55:27,666 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1110390962:90 , srv1152188336:90 , srv1153847676:90 , srv1181468868:90 , srv1251151497:90 , srv1253969260:90 , srv1293105913:90 , srv1333309623:90 , srv1392973777:90 , srv1415177715:90 , srv1464985163:90 , srv1528523691:90 , srv1581583791:90 , srv1581996072:90 , srv176334624:90 , srv1783407394:90 , srv1816195644:90 , srv1922901752:90 , srv1948761277:90 , srv262563110:90 , srv29515365:90 , srv394145958:90 , srv403365334:90 , srv610343380:90 , srv640380137:90 , srv660159197:90 , srv691183524:90 , srv751818423:90 , srv857671118:90 , srv881690727:90 }
2023-05-28 23:55:27,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-28 23:55:27,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv29515365=20, srv881690727=29, srv1251151497=4, srv1253969260=5, srv1392973777=8, srv1528523691=11, srv751818423=27, srv660159197=25, srv1152188336=1, srv403365334=22, srv610343380=23, srv857671118=28, srv1464985163=10, srv1816195644=16, srv262563110=19, srv1153847676=2, srv1333309623=7, srv1581996072=13, srv1922901752=17, srv176334624=14, srv640380137=24, srv1110390962=0, srv691183524=26, srv1415177715=9, srv1948761277=18, srv1783407394=15, srv1181468868=3, srv394145958=21, srv1293105913=6, srv1581583791=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-28 23:55:27,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-28 23:55:27,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-28 23:55:27,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-28 23:55:27,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-28 23:55:27,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-28 23:55:27,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-28 23:55:27,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-28 23:55:27,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-28 23:55:27,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-28 23:55:27,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-28 23:55:27,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-28 23:55:27,684 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-28 23:55:27,685 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1110390962:90 , srv1152188336:90 , srv1153847676:90 , srv1181468868:90 , srv1251151497:90 , srv1253969260:90 , srv1293105913:90 , srv1333309623:90 , srv1392973777:90 , srv1415177715:90 , srv1464985163:90 , srv1528523691:90 , srv1581583791:90 , srv1581996072:90 , srv176334624:90 , srv1783407394:90 , srv1816195644:90 , srv1922901752:90 , srv1948761277:90 , srv262563110:90 , srv29515365:90 , srv394145958:90 , srv403365334:90 , srv610343380:90 , srv640380137:90 , srv660159197:90 , srv691183524:90 , srv751818423:90 , srv857671118:90 , srv881690727:90 }
2023-05-28 23:55:27,697 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=214 (was 271), ProcessCount=172 (was 172), AvailableMemoryMB=4287 (was 4538)