2023-11-25 11:53:13,940 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-25 11:53:14,263 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-25 11:53:14,379 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-25 11:53:14,449 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-25 11:53:14,456 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-25 11:53:14,472 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=259, ProcessCount=171, AvailableMemoryMB=9734
2023-11-25 11:53:14,474 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-25 11:53:14,475 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-25 11:53:14,611 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000701505:84 , srv1047341176:84 , srv1059074673:84 , srv1074904173:84 , srv1095284127:84 , srv1099644075:84 , srv1119043595:84 , srv1136222615:84 , srv1158748243:84 , srv1172650560:84 , srv1174835861:84 , srv1222530541:84 , srv1226685712:84 , srv1230062404:84 , srv1278237231:84 , srv1299792958:84 , srv1302548398:84 , srv1322346547:84 , srv1328522714:84 , srv1425382411:84 , srv1463659183:84 , srv1471092806:84 , srv1474722725:84 , srv1481071649:84 , srv1512079877:84 , srv152207392:84 , srv1524205806:84 , srv1547631746:84 , srv1563622110:84 , srv1601868079:84 , srv1630734115:84 , srv1633769011:84 , srv1687436939:84 , srv1691931591:84 , srv1717061287:84 , srv1758301550:84 , srv1768403446:84 , srv1771812495:84 , srv1779856896:84 , srv1795082507:84 , srv1814481801:84 , srv1875673897:84 , srv1885713956:84 , srv1910378074:84 , srv1921119355:84 , srv1922903185:84 , srv1937162345:84 , srv1974700877:84 , srv1976084579:84 , srv2033393283:84 , srv2090708098:84 , srv2095016325:84 , srv2137079760:84 , srv223419500:84 , srv264739623:84 , srv271290149:84 , srv275094103:84 , srv298594115:84 , srv308137883:84 , srv308375275:84 , srv318958632:84 , srv334475411:84 , srv342006531:84 , srv351063597:84 , srv365595254:84 , srv375936427:84 , srv418793312:84 , srv437565550:84 , srv441797824:84 , srv45598904:84 , srv460673012:84 , srv462857514:84 , srv474945858:84 , srv499403133:84 , srv536958945:84 , srv54927544:84 , srv563700570:84 , srv633258175:84 , srv639556341:84 , srv641277487:84 , srv660467656:84 , srv668284136:84 , srv679213347:84 , srv706335355:84 , srv71981411:84 , srv748378897:84 , srv74865392:84 , srv766763647:84 , srv76885639:84 , srv790115172:84 , srv832302573:84 , srv833561741:84 , srv845178529:84 , srv849123688:84 , srv896264240:84 , srv911725724:84 , srv914256520:84 , srv929521512:84 , srv953155073:84 , srv672025978:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-25 11:53:14,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 11:53:14,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1136222615=7, srv1814481801=40, srv1885713956=42, srv342006531=62, srv1226685712=12, srv1795082507=39, srv1047341176=1, srv641277487=79, srv668284136=81, srv1230062404=13, srv1425382411=19, srv152207392=25, srv1633769011=31, srv318958632=60, srv639556341=78, srv308375275=59, srv334475411=61, srv633258175=77, srv1524205806=26, srv706335355=84, srv45598904=69, srv1474722725=22, srv1481071649=23, srv1512079877=24, srv1563622110=28, srv1717061287=34, srv1119043595=6, srv1158748243=8, srv1937162345=46, srv1074904173=3, srv1099644075=5, srv275094103=56, srv672025978=82, srv766763647=88, srv375936427=65, srv748378897=86, srv271290149=55, srv2033393283=49, srv790115172=90, srv1910378074=43, srv1222530541=11, srv1059074673=2, srv832302573=91, srv351063597=63, srv833561741=92, srv1278237231=14, srv1463659183=20, srv460673012=70, srv74865392=87, srv2137079760=52, srv223419500=53, srv679213347=83, srv563700570=76, srv845178529=93, srv418793312=66, srv911725724=96, srv1921119355=44, srv1976084579=48, srv1630734115=30, srv474945858=72, srv462857514=71, srv1095284127=4, srv1174835861=10, srv71981411=85, srv1922903185=45, srv54927544=75, srv896264240=95, srv499403133=73, srv264739623=54, srv1172650560=9, srv1601868079=29, srv1758301550=35, srv1687436939=32, srv1779856896=38, srv536958945=74, srv2095016325=51, srv953155073=99, srv1768403446=36, srv849123688=94, srv929521512=98, srv2090708098=50, srv365595254=64, srv1299792958=15, srv441797824=68, srv1328522714=18, srv1547631746=27, srv1974700877=47, srv308137883=58, srv1771812495=37, srv1471092806=21, srv1000701505=0, srv437565550=67, srv914256520=97, srv1691931591=33, srv298594115=57, srv76885639=89, srv1875673897=41, srv660467656=80, srv1302548398=16, srv1322346547=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 11:53:14,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 11:53:14,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 11:53:14,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 11:53:14,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 11:53:14,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 11:53:14,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 11:53:14,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 11:53:14,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 11:53:14,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 11:53:14,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 11:53:14,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 11:53:14,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 11:53:14,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 11:53:14,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 11:53:14,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 11:53:14,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 11:53:14,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 11:53:14,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 11:53:14,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 11:53:14,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 11:53:14,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 11:53:14,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 11:53:14,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 11:53:14,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 11:53:14,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 11:53:14,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 11:53:14,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 11:53:14,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 11:53:14,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 11:53:14,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 11:53:14,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-25 11:53:14,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-25 11:53:14,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-25 11:53:14,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-25 11:53:14,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-25 11:53:14,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-25 11:53:14,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-25 11:53:14,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-25 11:53:14,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-25 11:53:14,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-25 11:53:14,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-25 11:53:14,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-25 11:53:14,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-25 11:53:14,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-25 11:53:14,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-25 11:53:14,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-25 11:53:14,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-25 11:53:14,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-25 11:53:14,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-25 11:53:14,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-25 11:53:14,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-25 11:53:14,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-25 11:53:14,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-25 11:53:14,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-25 11:53:14,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-25 11:53:14,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-25 11:53:14,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-25 11:53:14,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-25 11:53:14,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-25 11:53:14,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-25 11:53:14,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-25 11:53:14,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-25 11:53:14,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-25 11:53:14,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-25 11:53:14,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-25 11:53:14,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-25 11:53:14,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-25 11:53:14,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-25 11:53:14,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-25 11:53:14,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-25 11:53:14,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-25 11:53:14,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-25 11:53:14,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-25 11:53:14,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-25 11:53:14,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-25 11:53:14,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-25 11:53:14,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-25 11:53:14,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-25 11:53:14,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-25 11:53:14,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-25 11:53:14,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-25 11:53:14,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-25 11:53:14,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-25 11:53:14,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-25 11:53:14,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-25 11:53:14,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-25 11:53:14,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-25 11:53:14,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-25 11:53:14,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-25 11:53:14,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-25 11:53:14,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-25 11:53:14,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-25 11:53:14,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-25 11:53:14,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-25 11:53:14,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-25 11:53:14,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-25 11:53:14,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-25 11:53:14,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-25 11:53:14,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-25 11:53:14,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-25 11:53:14,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-25 11:53:15,125 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:15,125 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-25 11:53:15,126 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-25 11:53:15,591 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 811 ms to try 5000 different iterations.  Found a solution that moves 4699 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17727489691166295. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04149320421177508, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055620164451417195, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5221111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04149320421177508, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17241666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23958333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:15,602 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv71981411:81 , srv790115172:81 , srv845178529:81 , srv1136222615:82 , srv2095016325:82 , srv914256520:82 , srv1158748243:83 , srv1172650560:83 , srv1474722725:83 , srv1910378074:83 , srv2137079760:83 , srv418793312:83 , srv441797824:83 , srv833561741:83 , srv1230062404:84 , srv1425382411:84 , srv298594115:84 , srv45598904:84 , srv460673012:84 , srv706335355:84 , srv74865392:84 , srv1299792958:85 , srv1322346547:85 , srv1481071649:85 , srv1512079877:85 , srv1691931591:85 , srv1875673897:85 , srv1974700877:85 , srv2033393283:85 , srv474945858:85 , srv499403133:85 , srv54927544:85 , srv832302573:85 , srv911725724:85 , srv1226685712:86 , srv1278237231:86 , srv1302548398:86 , srv1463659183:86 , srv152207392:86 , srv1524205806:86 , srv1630734115:86 , srv1717061287:86 , srv1795082507:86 , srv1976084579:86 , srv318958632:86 , srv365595254:86 , srv633258175:86 , srv766763647:86 , srv1095284127:87 , srv1099644075:87 , srv1174835861:87 , srv1328522714:87 , srv1471092806:87 , srv1547631746:87 , srv1563622110:87 , srv1758301550:87 , srv1771812495:87 , srv1779856896:87 , srv1885713956:87 , srv1921119355:87 , srv1937162345:87 , srv223419500:87 , srv264739623:87 , srv308375275:87 , srv334475411:87 , srv342006531:87 , srv375936427:87 , srv462857514:87 , srv536958945:87 , srv76885639:87 , srv929521512:87 , srv1119043595:88 , srv1222530541:88 , srv1601868079:88 , srv1687436939:88 , srv1814481801:88 , srv1922903185:88 , srv275094103:88 , srv351063597:88 , srv563700570:88 , srv641277487:88 , srv668284136:88 , srv679213347:88 , srv1047341176:89 , srv1633769011:89 , srv1768403446:89 , srv2090708098:89 , srv308137883:89 , srv437565550:89 , srv639556341:89 , srv660467656:89 , srv748378897:89 , srv896264240:89 , srv953155073:89 , srv271290149:90 , srv1000701505:91 , srv1059074673:91 , srv849123688:91 , srv1074904173:92 , srv672025978:459 }
2023-11-25 11:53:15,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 11:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1136222615=7, srv1814481801=40, srv1885713956=42, srv342006531=62, srv1226685712=12, srv1795082507=39, srv1047341176=1, srv641277487=79, srv668284136=81, srv1230062404=13, srv1425382411=19, srv152207392=25, srv1633769011=31, srv318958632=60, srv639556341=78, srv308375275=59, srv334475411=61, srv633258175=77, srv1524205806=26, srv706335355=84, srv45598904=69, srv1474722725=22, srv1481071649=23, srv1512079877=24, srv1563622110=28, srv1717061287=34, srv1119043595=6, srv1158748243=8, srv1937162345=46, srv1074904173=3, srv1099644075=5, srv275094103=56, srv672025978=82, srv766763647=88, srv375936427=65, srv748378897=86, srv271290149=55, srv2033393283=49, srv790115172=90, srv1910378074=43, srv1222530541=11, srv1059074673=2, srv832302573=91, srv351063597=63, srv833561741=92, srv1278237231=14, srv1463659183=20, srv460673012=70, srv74865392=87, srv2137079760=52, srv223419500=53, srv679213347=83, srv563700570=76, srv845178529=93, srv418793312=66, srv911725724=96, srv1921119355=44, srv1976084579=48, srv1630734115=30, srv474945858=72, srv462857514=71, srv1095284127=4, srv1174835861=10, srv71981411=85, srv1922903185=45, srv54927544=75, srv896264240=95, srv499403133=73, srv264739623=54, srv1172650560=9, srv1601868079=29, srv1758301550=35, srv1687436939=32, srv1779856896=38, srv536958945=74, srv2095016325=51, srv953155073=99, srv1768403446=36, srv849123688=94, srv929521512=98, srv2090708098=50, srv365595254=64, srv1299792958=15, srv441797824=68, srv1328522714=18, srv1547631746=27, srv1974700877=47, srv308137883=58, srv1771812495=37, srv1471092806=21, srv1000701505=0, srv437565550=67, srv914256520=97, srv1691931591=33, srv298594115=57, srv76885639=89, srv1875673897=41, srv660467656=80, srv1302548398=16, srv1322346547=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 11:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 11:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 11:53:15,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-25 11:53:15,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-25 11:53:15,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-25 11:53:15,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-25 11:53:15,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-25 11:53:15,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-25 11:53:15,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-25 11:53:15,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-25 11:53:15,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-25 11:53:15,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-25 11:53:15,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-25 11:53:15,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-25 11:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-25 11:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-25 11:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-25 11:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-25 11:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-25 11:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-25 11:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-25 11:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-25 11:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-25 11:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-25 11:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-25 11:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-25 11:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-25 11:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-25 11:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-25 11:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-25 11:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-25 11:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-25 11:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-25 11:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-25 11:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-25 11:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-25 11:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-25 11:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-25 11:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-25 11:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-25 11:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-25 11:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-25 11:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-25 11:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-25 11:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-25 11:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-25 11:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-25 11:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-25 11:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-25 11:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-25 11:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-25 11:53:15,778 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04149320421177508, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055620164451417195, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04149320421177508, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17241666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23958333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:15,779 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-25 11:53:15,779 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17724199381041086, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04149320421177508, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055620164451417195, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04149320421177508, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17241666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23958333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-25 11:53:16,118 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 463 ms to try 5000 different iterations.  Found a solution that moves 2649 regions; Going from a computed imbalance of 0.17724199381041086 to a new imbalance of 0.014577341635270207. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02409918400401886); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033507960090850886, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29433333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02409918400401886); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:16,129 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1074904173:79 , srv1059074673:80 , srv437565550:80 , srv639556341:80 , srv748378897:80 , srv896264240:80 , srv1000701505:81 , srv1047341176:81 , srv271290149:81 , srv275094103:81 , srv1222530541:82 , srv1758301550:82 , srv2090708098:82 , srv308137883:82 , srv351063597:82 , srv679213347:82 , srv849123688:82 , srv1226685712:83 , srv1633769011:83 , srv1687436939:83 , srv1768403446:83 , srv342006531:83 , srv462857514:83 , srv563700570:83 , srv641277487:83 , srv660467656:83 , srv1885713956:84 , srv1937162345:84 , srv2033393283:84 , srv375936427:84 , srv536958945:84 , srv1099644075:85 , srv1174835861:85 , srv1278237231:85 , srv1299792958:85 , srv1328522714:85 , srv1524205806:85 , srv1547631746:85 , srv1717061287:85 , srv1779856896:85 , srv1922903185:85 , srv223419500:85 , srv264739623:85 , srv460673012:85 , srv668284136:85 , srv766763647:85 , srv76885639:85 , srv1322346547:86 , srv1471092806:86 , srv1512079877:86 , srv152207392:86 , srv1601868079:86 , srv1630734115:86 , srv1771812495:86 , srv1795082507:86 , srv441797824:86 , srv474945858:86 , srv914256520:86 , srv953155073:86 , srv1158748243:87 , srv1463659183:87 , srv1563622110:87 , srv1921119355:87 , srv1974700877:87 , srv1976084579:87 , srv308375275:87 , srv334475411:87 , srv365595254:87 , srv633258175:87 , srv929521512:87 , srv1095284127:88 , srv1136222615:88 , srv1302548398:88 , srv1425382411:88 , srv1474722725:88 , srv1691931591:88 , srv1814481801:88 , srv2137079760:88 , srv318958632:88 , srv45598904:88 , srv499403133:88 , srv54927544:88 , srv706335355:88 , srv71981411:88 , srv833561741:88 , srv911725724:88 , srv1119043595:89 , srv1172650560:89 , srv1230062404:89 , srv1481071649:89 , srv1875673897:89 , srv418793312:89 , srv832302573:89 , srv2095016325:90 , srv298594115:90 , srv74865392:91 , srv845178529:91 , srv1910378074:92 , srv790115172:93 , srv672025978:529 }
2023-11-25 11:53:16,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 11:53:16,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1136222615=7, srv1814481801=40, srv1885713956=42, srv342006531=62, srv1226685712=12, srv1795082507=39, srv1047341176=1, srv641277487=79, srv668284136=81, srv1230062404=13, srv1425382411=19, srv152207392=25, srv1633769011=31, srv318958632=60, srv639556341=78, srv308375275=59, srv334475411=61, srv633258175=77, srv1524205806=26, srv706335355=84, srv45598904=69, srv1474722725=22, srv1481071649=23, srv1512079877=24, srv1563622110=28, srv1717061287=34, srv1119043595=6, srv1158748243=8, srv1937162345=46, srv1074904173=3, srv1099644075=5, srv275094103=56, srv672025978=82, srv766763647=88, srv375936427=65, srv748378897=86, srv271290149=55, srv2033393283=49, srv790115172=90, srv1910378074=43, srv1222530541=11, srv1059074673=2, srv832302573=91, srv351063597=63, srv833561741=92, srv1278237231=14, srv1463659183=20, srv460673012=70, srv74865392=87, srv2137079760=52, srv223419500=53, srv679213347=83, srv563700570=76, srv845178529=93, srv418793312=66, srv911725724=96, srv1921119355=44, srv1976084579=48, srv1630734115=30, srv474945858=72, srv462857514=71, srv1095284127=4, srv1174835861=10, srv71981411=85, srv1922903185=45, srv54927544=75, srv896264240=95, srv499403133=73, srv264739623=54, srv1172650560=9, srv1601868079=29, srv1758301550=35, srv1687436939=32, srv1779856896=38, srv536958945=74, srv2095016325=51, srv953155073=99, srv1768403446=36, srv849123688=94, srv929521512=98, srv2090708098=50, srv365595254=64, srv1299792958=15, srv441797824=68, srv1328522714=18, srv1547631746=27, srv1974700877=47, srv308137883=58, srv1771812495=37, srv1471092806=21, srv1000701505=0, srv437565550=67, srv914256520=97, srv1691931591=33, srv298594115=57, srv76885639=89, srv1875673897=41, srv660467656=80, srv1302548398=16, srv1322346547=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 11:53:16,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 11:53:16,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 11:53:16,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 11:53:16,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 11:53:16,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 11:53:16,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 11:53:16,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 11:53:16,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 11:53:16,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 11:53:16,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-25 11:53:16,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-25 11:53:16,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-25 11:53:16,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-25 11:53:16,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-25 11:53:16,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-25 11:53:16,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-25 11:53:16,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-25 11:53:16,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-25 11:53:16,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-25 11:53:16,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-25 11:53:16,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-25 11:53:16,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-25 11:53:16,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-25 11:53:16,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-25 11:53:16,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-25 11:53:16,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-25 11:53:16,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-25 11:53:16,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-25 11:53:16,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-25 11:53:16,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-25 11:53:16,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-25 11:53:16,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-25 11:53:16,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-25 11:53:16,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-25 11:53:16,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-25 11:53:16,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-25 11:53:16,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-25 11:53:16,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-25 11:53:16,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-25 11:53:16,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-25 11:53:16,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-25 11:53:16,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-25 11:53:16,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-25 11:53:16,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-25 11:53:16,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-25 11:53:16,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-25 11:53:16,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-25 11:53:16,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-25 11:53:16,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-25 11:53:16,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-25 11:53:16,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-25 11:53:16,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-25 11:53:16,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-25 11:53:16,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-25 11:53:16,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-25 11:53:16,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-25 11:53:16,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-25 11:53:16,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-25 11:53:16,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-25 11:53:16,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-25 11:53:16,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-25 11:53:16,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-25 11:53:16,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-25 11:53:16,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-25 11:53:16,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-25 11:53:16,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-25 11:53:16,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-25 11:53:16,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-25 11:53:16,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-25 11:53:16,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-25 11:53:16,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-25 11:53:16,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-25 11:53:16,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-25 11:53:16,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-25 11:53:16,321 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02409918400401886); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033507960090850886, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02409918400401886); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:16,321 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-25 11:53:16,322 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014558792940821012, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02409918400401886); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033507960090850886, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02409918400401886); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-25 11:53:16,610 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 409 ms to try 5000 different iterations.  Found a solution that moves 924 regions; Going from a computed imbalance of 0.014558792940821012 to a new imbalance of 7.029491813160096E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007547496857671467); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009983150788368353); MoveCostFunction : (multiplier=7.0, imbalance=0.10266666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007547496857671467); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:16,613 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1119043595:81 , srv1814481801:81 , srv1095284127:82 , srv74865392:82 , srv1481071649:83 , srv1601868079:83 , srv1875673897:83 , srv1910378074:83 , srv298594115:83 , srv308375275:83 , srv790115172:83 , srv929521512:83 , srv953155073:83 , srv1230062404:84 , srv1302548398:84 , srv1563622110:84 , srv1691931591:84 , srv1771812495:84 , srv1921119355:84 , srv1922903185:84 , srv2095016325:84 , srv334475411:84 , srv45598904:84 , srv668284136:84 , srv832302573:84 , srv849123688:84 , srv1000701505:85 , srv1172650560:85 , srv1174835861:85 , srv1322346547:85 , srv1328522714:85 , srv1425382411:85 , srv1463659183:85 , srv1471092806:85 , srv152207392:85 , srv1547631746:85 , srv1630734115:85 , srv1633769011:85 , srv1717061287:85 , srv1768403446:85 , srv1779856896:85 , srv1974700877:85 , srv1976084579:85 , srv264739623:85 , srv318958632:85 , srv365595254:85 , srv418793312:85 , srv499403133:85 , srv54927544:85 , srv633258175:85 , srv706335355:85 , srv76885639:85 , srv845178529:85 , srv911725724:85 , srv1059074673:86 , srv1074904173:86 , srv1099644075:86 , srv1278237231:86 , srv1474722725:86 , srv1524205806:86 , srv1687436939:86 , srv1795082507:86 , srv1885713956:86 , srv1937162345:86 , srv2090708098:86 , srv2137079760:86 , srv223419500:86 , srv308137883:86 , srv375936427:86 , srv474945858:86 , srv536958945:86 , srv641277487:86 , srv660467656:86 , srv766763647:86 , srv833561741:86 , srv1047341176:87 , srv1222530541:87 , srv1299792958:87 , srv1512079877:87 , srv271290149:87 , srv342006531:87 , srv351063597:87 , srv1136222615:88 , srv1158748243:88 , srv1226685712:88 , srv275094103:88 , srv462857514:88 , srv563700570:88 , srv639556341:88 , srv679213347:88 , srv748378897:88 , srv896264240:88 , srv1758301550:89 , srv2033393283:89 , srv437565550:89 , srv441797824:89 , srv460673012:89 , srv71981411:89 , srv914256520:89 , srv672025978:537 }
2023-11-25 11:53:16,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 11:53:16,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1136222615=7, srv1814481801=40, srv1885713956=42, srv342006531=62, srv1226685712=12, srv1795082507=39, srv1047341176=1, srv641277487=79, srv668284136=81, srv1230062404=13, srv1425382411=19, srv152207392=25, srv1633769011=31, srv318958632=60, srv639556341=78, srv308375275=59, srv334475411=61, srv633258175=77, srv1524205806=26, srv706335355=84, srv45598904=69, srv1474722725=22, srv1481071649=23, srv1512079877=24, srv1563622110=28, srv1717061287=34, srv1119043595=6, srv1158748243=8, srv1937162345=46, srv1074904173=3, srv1099644075=5, srv275094103=56, srv672025978=82, srv766763647=88, srv375936427=65, srv748378897=86, srv271290149=55, srv2033393283=49, srv790115172=90, srv1910378074=43, srv1222530541=11, srv1059074673=2, srv832302573=91, srv351063597=63, srv833561741=92, srv1278237231=14, srv1463659183=20, srv460673012=70, srv74865392=87, srv2137079760=52, srv223419500=53, srv679213347=83, srv563700570=76, srv845178529=93, srv418793312=66, srv911725724=96, srv1921119355=44, srv1976084579=48, srv1630734115=30, srv474945858=72, srv462857514=71, srv1095284127=4, srv1174835861=10, srv71981411=85, srv1922903185=45, srv54927544=75, srv896264240=95, srv499403133=73, srv264739623=54, srv1172650560=9, srv1601868079=29, srv1758301550=35, srv1687436939=32, srv1779856896=38, srv536958945=74, srv2095016325=51, srv953155073=99, srv1768403446=36, srv849123688=94, srv929521512=98, srv2090708098=50, srv365595254=64, srv1299792958=15, srv441797824=68, srv1328522714=18, srv1547631746=27, srv1974700877=47, srv308137883=58, srv1771812495=37, srv1471092806=21, srv1000701505=0, srv437565550=67, srv914256520=97, srv1691931591=33, srv298594115=57, srv76885639=89, srv1875673897=41, srv660467656=80, srv1302548398=16, srv1322346547=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 11:53:16,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 11:53:16,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 11:53:16,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 11:53:16,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 11:53:16,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 11:53:16,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 11:53:16,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 11:53:16,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 11:53:16,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 11:53:16,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 11:53:16,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 11:53:16,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 11:53:16,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 11:53:16,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 11:53:16,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 11:53:16,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 11:53:16,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 11:53:16,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-25 11:53:16,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-25 11:53:16,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-25 11:53:16,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-25 11:53:16,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-25 11:53:16,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-25 11:53:16,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-25 11:53:16,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-25 11:53:16,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-25 11:53:16,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-25 11:53:16,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-25 11:53:16,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-25 11:53:16,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-25 11:53:16,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-25 11:53:16,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-25 11:53:16,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-25 11:53:16,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-25 11:53:16,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-25 11:53:16,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-25 11:53:16,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-25 11:53:16,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-25 11:53:16,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-25 11:53:16,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-25 11:53:16,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-25 11:53:16,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-25 11:53:16,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-25 11:53:16,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-25 11:53:16,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-25 11:53:16,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-25 11:53:16,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-25 11:53:16,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-25 11:53:16,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-25 11:53:16,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-25 11:53:16,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-25 11:53:16,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-25 11:53:16,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-25 11:53:16,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-25 11:53:16,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-25 11:53:16,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-25 11:53:16,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-25 11:53:16,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-25 11:53:16,774 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007547496857671467); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009983150788368353); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007547496857671467); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:16,774 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-25 11:53:16,775 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.964791950302199E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007547496857671467); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009983150788368353); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007547496857671467); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-25 11:53:17,191 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 542 ms to try 5000 different iterations.  Found a solution that moves 249 regions; Going from a computed imbalance of 6.964791950302199E-4 to a new imbalance of 3.5697943966298215E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.027666666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:17,193 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv318958632:83 , srv460673012:83 , srv563700570:83 , srv1099644075:84 , srv1119043595:84 , srv1136222615:84 , srv1158748243:84 , srv1174835861:84 , srv1302548398:84 , srv1471092806:84 , srv1474722725:84 , srv1512079877:84 , srv1563622110:84 , srv1795082507:84 , srv1885713956:84 , srv1910378074:84 , srv1921119355:84 , srv2033393283:84 , srv223419500:84 , srv271290149:84 , srv275094103:84 , srv334475411:84 , srv342006531:84 , srv365595254:84 , srv418793312:84 , srv437565550:84 , srv462857514:84 , srv499403133:84 , srv54927544:84 , srv633258175:84 , srv660467656:84 , srv679213347:84 , srv766763647:84 , srv832302573:84 , srv953155073:84 , srv1000701505:85 , srv1047341176:85 , srv1059074673:85 , srv1074904173:85 , srv1095284127:85 , srv1172650560:85 , srv1222530541:85 , srv1226685712:85 , srv1230062404:85 , srv1278237231:85 , srv1299792958:85 , srv1328522714:85 , srv1425382411:85 , srv1463659183:85 , srv1481071649:85 , srv152207392:85 , srv1524205806:85 , srv1547631746:85 , srv1601868079:85 , srv1630734115:85 , srv1633769011:85 , srv1687436939:85 , srv1691931591:85 , srv1758301550:85 , srv1768403446:85 , srv1779856896:85 , srv1814481801:85 , srv1875673897:85 , srv1922903185:85 , srv1937162345:85 , srv1974700877:85 , srv1976084579:85 , srv2090708098:85 , srv2137079760:85 , srv264739623:85 , srv298594115:85 , srv308137883:85 , srv308375275:85 , srv351063597:85 , srv375936427:85 , srv441797824:85 , srv474945858:85 , srv536958945:85 , srv639556341:85 , srv641277487:85 , srv668284136:85 , srv706335355:85 , srv71981411:85 , srv748378897:85 , srv74865392:85 , srv76885639:85 , srv790115172:85 , srv833561741:85 , srv845178529:85 , srv849123688:85 , srv896264240:85 , srv911725724:85 , srv914256520:85 , srv929521512:85 , srv1322346547:86 , srv1771812495:86 , srv2095016325:86 , srv45598904:86 , srv1717061287:87 , srv672025978:617 }
2023-11-25 11:53:17,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 11:53:17,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1136222615=7, srv1814481801=40, srv1885713956=42, srv342006531=62, srv1226685712=12, srv1795082507=39, srv1047341176=1, srv641277487=79, srv668284136=81, srv1230062404=13, srv1425382411=19, srv152207392=25, srv1633769011=31, srv318958632=60, srv639556341=78, srv308375275=59, srv334475411=61, srv633258175=77, srv1524205806=26, srv706335355=84, srv45598904=69, srv1474722725=22, srv1481071649=23, srv1512079877=24, srv1563622110=28, srv1717061287=34, srv1119043595=6, srv1158748243=8, srv1937162345=46, srv1074904173=3, srv1099644075=5, srv275094103=56, srv672025978=82, srv766763647=88, srv375936427=65, srv748378897=86, srv271290149=55, srv2033393283=49, srv790115172=90, srv1910378074=43, srv1222530541=11, srv1059074673=2, srv832302573=91, srv351063597=63, srv833561741=92, srv1278237231=14, srv1463659183=20, srv460673012=70, srv74865392=87, srv2137079760=52, srv223419500=53, srv679213347=83, srv563700570=76, srv845178529=93, srv418793312=66, srv911725724=96, srv1921119355=44, srv1976084579=48, srv1630734115=30, srv474945858=72, srv462857514=71, srv1095284127=4, srv1174835861=10, srv71981411=85, srv1922903185=45, srv54927544=75, srv896264240=95, srv499403133=73, srv264739623=54, srv1172650560=9, srv1601868079=29, srv1758301550=35, srv1687436939=32, srv1779856896=38, srv536958945=74, srv2095016325=51, srv953155073=99, srv1768403446=36, srv849123688=94, srv929521512=98, srv2090708098=50, srv365595254=64, srv1299792958=15, srv441797824=68, srv1328522714=18, srv1547631746=27, srv1974700877=47, srv308137883=58, srv1771812495=37, srv1471092806=21, srv1000701505=0, srv437565550=67, srv914256520=97, srv1691931591=33, srv298594115=57, srv76885639=89, srv1875673897=41, srv660467656=80, srv1302548398=16, srv1322346547=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 11:53:17,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 11:53:17,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 11:53:17,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 11:53:17,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 11:53:17,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-25 11:53:17,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-25 11:53:17,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-25 11:53:17,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-25 11:53:17,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-25 11:53:17,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-25 11:53:17,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-25 11:53:17,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-25 11:53:17,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-25 11:53:17,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-25 11:53:17,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-25 11:53:17,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-25 11:53:17,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-25 11:53:17,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-25 11:53:17,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-25 11:53:17,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-25 11:53:17,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-25 11:53:17,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-25 11:53:17,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-25 11:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-25 11:53:17,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-25 11:53:17,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-25 11:53:17,333 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:17,333 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-25 11:53:17,334 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.395440870097176E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-25 11:53:17,700 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 472 ms to try 5000 different iterations.  Found a solution that moves 32 regions; Going from a computed imbalance of 3.395440870097176E-5 to a new imbalance of 6.968137842950207E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0035555555555555557); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:17,701 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1463659183:83 , srv1717061287:83 , srv1758301550:83 , srv1771812495:83 , srv1976084579:83 , srv441797824:83 , srv71981411:83 , srv911725724:83 , srv1000701505:84 , srv1047341176:84 , srv1059074673:84 , srv1074904173:84 , srv1095284127:84 , srv1099644075:84 , srv1119043595:84 , srv1136222615:84 , srv1158748243:84 , srv1172650560:84 , srv1222530541:84 , srv1226685712:84 , srv1230062404:84 , srv1278237231:84 , srv1299792958:84 , srv1302548398:84 , srv1322346547:84 , srv1328522714:84 , srv1425382411:84 , srv1471092806:84 , srv1481071649:84 , srv1512079877:84 , srv152207392:84 , srv1524205806:84 , srv1547631746:84 , srv1563622110:84 , srv1601868079:84 , srv1630734115:84 , srv1633769011:84 , srv1687436939:84 , srv1691931591:84 , srv1768403446:84 , srv1779856896:84 , srv1795082507:84 , srv1814481801:84 , srv1875673897:84 , srv1910378074:84 , srv1921119355:84 , srv1922903185:84 , srv1937162345:84 , srv1974700877:84 , srv2033393283:84 , srv2090708098:84 , srv2095016325:84 , srv2137079760:84 , srv223419500:84 , srv264739623:84 , srv271290149:84 , srv298594115:84 , srv308137883:84 , srv308375275:84 , srv318958632:84 , srv334475411:84 , srv351063597:84 , srv365595254:84 , srv375936427:84 , srv437565550:84 , srv45598904:84 , srv462857514:84 , srv474945858:84 , srv499403133:84 , srv536958945:84 , srv54927544:84 , srv563700570:84 , srv633258175:84 , srv639556341:84 , srv641277487:84 , srv660467656:84 , srv668284136:84 , srv679213347:84 , srv706335355:84 , srv748378897:84 , srv74865392:84 , srv76885639:84 , srv790115172:84 , srv832302573:84 , srv833561741:84 , srv845178529:84 , srv849123688:84 , srv896264240:84 , srv914256520:84 , srv929521512:84 , srv953155073:84 , srv1174835861:85 , srv1474722725:85 , srv1885713956:85 , srv275094103:85 , srv342006531:85 , srv418793312:85 , srv460673012:85 , srv766763647:85 , srv672025978:684 }
2023-11-25 11:53:17,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 11:53:17,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1136222615=7, srv1814481801=40, srv1885713956=42, srv342006531=62, srv1226685712=12, srv1795082507=39, srv1047341176=1, srv641277487=79, srv668284136=81, srv1230062404=13, srv1425382411=19, srv152207392=25, srv1633769011=31, srv318958632=60, srv639556341=78, srv308375275=59, srv334475411=61, srv633258175=77, srv1524205806=26, srv706335355=84, srv45598904=69, srv1474722725=22, srv1481071649=23, srv1512079877=24, srv1563622110=28, srv1717061287=34, srv1119043595=6, srv1158748243=8, srv1937162345=46, srv1074904173=3, srv1099644075=5, srv275094103=56, srv672025978=82, srv766763647=88, srv375936427=65, srv748378897=86, srv271290149=55, srv2033393283=49, srv790115172=90, srv1910378074=43, srv1222530541=11, srv1059074673=2, srv832302573=91, srv351063597=63, srv833561741=92, srv1278237231=14, srv1463659183=20, srv460673012=70, srv74865392=87, srv2137079760=52, srv223419500=53, srv679213347=83, srv563700570=76, srv845178529=93, srv418793312=66, srv911725724=96, srv1921119355=44, srv1976084579=48, srv1630734115=30, srv474945858=72, srv462857514=71, srv1095284127=4, srv1174835861=10, srv71981411=85, srv1922903185=45, srv54927544=75, srv896264240=95, srv499403133=73, srv264739623=54, srv1172650560=9, srv1601868079=29, srv1758301550=35, srv1687436939=32, srv1779856896=38, srv536958945=74, srv2095016325=51, srv953155073=99, srv1768403446=36, srv849123688=94, srv929521512=98, srv2090708098=50, srv365595254=64, srv1299792958=15, srv441797824=68, srv1328522714=18, srv1547631746=27, srv1974700877=47, srv308137883=58, srv1771812495=37, srv1471092806=21, srv1000701505=0, srv437565550=67, srv914256520=97, srv1691931591=33, srv298594115=57, srv76885639=89, srv1875673897=41, srv660467656=80, srv1302548398=16, srv1322346547=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 11:53:17,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 11:53:17,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 11:53:17,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 11:53:17,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-25 11:53:17,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-25 11:53:17,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-25 11:53:17,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-25 11:53:17,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-25 11:53:17,851 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:17,851 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1463659183:83 , srv1717061287:83 , srv1758301550:83 , srv1771812495:83 , srv1976084579:83 , srv441797824:83 , srv71981411:83 , srv911725724:83 , srv1000701505:84 , srv1047341176:84 , srv1059074673:84 , srv1074904173:84 , srv1095284127:84 , srv1099644075:84 , srv1119043595:84 , srv1136222615:84 , srv1158748243:84 , srv1172650560:84 , srv1222530541:84 , srv1226685712:84 , srv1230062404:84 , srv1278237231:84 , srv1299792958:84 , srv1302548398:84 , srv1322346547:84 , srv1328522714:84 , srv1425382411:84 , srv1471092806:84 , srv1481071649:84 , srv1512079877:84 , srv152207392:84 , srv1524205806:84 , srv1547631746:84 , srv1563622110:84 , srv1601868079:84 , srv1630734115:84 , srv1633769011:84 , srv1687436939:84 , srv1691931591:84 , srv1768403446:84 , srv1779856896:84 , srv1795082507:84 , srv1814481801:84 , srv1875673897:84 , srv1910378074:84 , srv1921119355:84 , srv1922903185:84 , srv1937162345:84 , srv1974700877:84 , srv2033393283:84 , srv2090708098:84 , srv2095016325:84 , srv2137079760:84 , srv223419500:84 , srv264739623:84 , srv271290149:84 , srv298594115:84 , srv308137883:84 , srv308375275:84 , srv318958632:84 , srv334475411:84 , srv351063597:84 , srv365595254:84 , srv375936427:84 , srv437565550:84 , srv45598904:84 , srv462857514:84 , srv474945858:84 , srv499403133:84 , srv536958945:84 , srv54927544:84 , srv563700570:84 , srv633258175:84 , srv639556341:84 , srv641277487:84 , srv660467656:84 , srv668284136:84 , srv679213347:84 , srv706335355:84 , srv748378897:84 , srv74865392:84 , srv76885639:84 , srv790115172:84 , srv832302573:84 , srv833561741:84 , srv845178529:84 , srv849123688:84 , srv896264240:84 , srv914256520:84 , srv929521512:84 , srv953155073:84 , srv1174835861:85 , srv1474722725:85 , srv1885713956:85 , srv275094103:85 , srv342006531:85 , srv418793312:85 , srv460673012:85 , srv766763647:85 , srv672025978:684 }
2023-11-25 11:53:17,882 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=278 (was 259) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=8819 (was 9734)
2023-11-25 11:53:17,892 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=278, ProcessCount=171, AvailableMemoryMB=8817
2023-11-25 11:53:17,893 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-25 11:53:17,893 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-25 11:53:17,902 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv110900573:84 , srv1167456182:84 , srv1300268020:84 , srv1317177845:84 , srv1342620442:84 , srv160806448:84 , srv1664642546:84 , srv1738727842:84 , srv1757658263:84 , srv1816956930:84 , srv1829568699:84 , srv1830093320:84 , srv1849688353:84 , srv1888526205:84 , srv1939670089:84 , srv1959637530:84 , srv2020840020:84 , srv237619734:84 , srv242291943:84 , srv358557557:84 , srv385320054:84 , srv416342028:84 , srv492069457:84 , srv5239659:84 , srv559190756:84 , srv66225394:84 , srv70173294:84 , srv803712451:84 , srv892402029:84 , srv163654814:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-25 11:53:17,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 11:53:17,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1849688353=13, srv160806448=5, srv492069457=23, srv5239659=24, srv66225394=26, srv1300268020=2, srv1939670089=15, srv1167456182=1, srv1829568699=11, srv358557557=20, srv1342620442=4, srv1757658263=9, srv242291943=19, srv2020840020=17, srv237619734=18, srv1664642546=7, srv163654814=6, srv1816956930=10, srv1317177845=3, srv1959637530=16, srv416342028=22, srv110900573=0, srv1738727842=8, srv1888526205=14, srv803712451=28, srv559190756=25, srv1830093320=12, srv892402029=29, srv70173294=27, srv385320054=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 11:53:17,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 11:53:17,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 11:53:17,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 11:53:17,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 11:53:17,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 11:53:17,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 11:53:17,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 11:53:17,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-25 11:53:17,941 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0993173139037777, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:53:17,941 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912505988877383, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0993173139037777, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-25 11:55:17,923 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5449425 different iterations.  Found a solution that moves 1853 regions; Going from a computed imbalance of 0.9912505988877383 to a new imbalance of 4.324994439959734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6862962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:55:17,929 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv110900573:90 , srv1167456182:90 , srv1300268020:90 , srv1317177845:90 , srv1342620442:90 , srv160806448:90 , srv163654814:90 , srv1664642546:90 , srv1738727842:90 , srv1757658263:90 , srv1816956930:90 , srv1829568699:90 , srv1830093320:90 , srv1849688353:90 , srv1888526205:90 , srv1939670089:90 , srv1959637530:90 , srv2020840020:90 , srv237619734:90 , srv242291943:90 , srv358557557:90 , srv385320054:90 , srv416342028:90 , srv492069457:90 , srv5239659:90 , srv559190756:90 , srv66225394:90 , srv70173294:90 , srv803712451:90 , srv892402029:90 }
2023-11-25 11:55:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 11:55:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1849688353=13, srv160806448=5, srv492069457=23, srv5239659=24, srv66225394=26, srv1300268020=2, srv1939670089=15, srv1167456182=1, srv1829568699=11, srv358557557=20, srv1342620442=4, srv1757658263=9, srv242291943=19, srv2020840020=17, srv237619734=18, srv1664642546=7, srv163654814=6, srv1816956930=10, srv1317177845=3, srv1959637530=16, srv416342028=22, srv110900573=0, srv1738727842=8, srv1888526205=14, srv803712451=28, srv559190756=25, srv1830093320=12, srv892402029=29, srv70173294=27, srv385320054=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 11:55:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 11:55:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-25 11:55:17,949 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 11:55:17,949 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv110900573:90 , srv1167456182:90 , srv1300268020:90 , srv1317177845:90 , srv1342620442:90 , srv160806448:90 , srv163654814:90 , srv1664642546:90 , srv1738727842:90 , srv1757658263:90 , srv1816956930:90 , srv1829568699:90 , srv1830093320:90 , srv1849688353:90 , srv1888526205:90 , srv1939670089:90 , srv1959637530:90 , srv2020840020:90 , srv237619734:90 , srv242291943:90 , srv358557557:90 , srv385320054:90 , srv416342028:90 , srv492069457:90 , srv5239659:90 , srv559190756:90 , srv66225394:90 , srv70173294:90 , srv803712451:90 , srv892402029:90 }
2023-11-25 11:55:17,961 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=153 (was 278), ProcessCount=171 (was 171), AvailableMemoryMB=8208 (was 8817)