2023-08-03 16:53:14,286 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-03 16:53:14,614 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-03 16:53:14,739 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-03 16:53:14,807 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-03 16:53:14,818 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-03 16:53:14,832 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=294, ProcessCount=175, AvailableMemoryMB=2640
2023-08-03 16:53:14,834 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-03 16:53:14,834 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-03 16:53:14,980 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1018108581:84 , srv1018486873:84 , srv105009291:84 , srv1065162553:84 , srv1065478879:84 , srv1083352573:84 , srv1094165374:84 , srv1109545094:84 , srv1117409009:84 , srv1179666073:84 , srv1228756369:84 , srv1232465744:84 , srv126571553:84 , srv1281645432:84 , srv1291020759:84 , srv1312578900:84 , srv1352235037:84 , srv1353623804:84 , srv1372707832:84 , srv1457184385:84 , srv1472926160:84 , srv1517659689:84 , srv156245486:84 , srv1580823119:84 , srv1581182755:84 , srv1628153859:84 , srv1673696929:84 , srv1698858602:84 , srv1703373458:84 , srv1705673046:84 , srv1706637487:84 , srv1715519832:84 , srv1718288072:84 , srv1723460186:84 , srv1740300804:84 , srv1740831814:84 , srv1747292536:84 , srv175862891:84 , srv1842719227:84 , srv1850187373:84 , srv1881642444:84 , srv1912877484:84 , srv1992594773:84 , srv1994950894:84 , srv2012081606:84 , srv2027142322:84 , srv20643178:84 , srv2069188445:84 , srv216155200:84 , srv216380191:84 , srv225828617:84 , srv233439362:84 , srv234430425:84 , srv243688819:84 , srv262732033:84 , srv272361757:84 , srv274902087:84 , srv294063743:84 , srv323430857:84 , srv335728627:84 , srv397499602:84 , srv440139438:84 , srv497203538:84 , srv501439699:84 , srv511253068:84 , srv526000162:84 , srv527467538:84 , srv536973908:84 , srv540419818:84 , srv545591806:84 , srv591992603:84 , srv594415757:84 , srv600914303:84 , srv616830885:84 , srv632303836:84 , srv649481986:84 , srv666167297:84 , srv67188395:84 , srv693463839:84 , srv730190265:84 , srv758171196:84 , srv769930033:84 , srv782667795:84 , srv788291095:84 , srv858883599:84 , srv866517197:84 , srv869465730:84 , srv871874184:84 , srv879533808:84 , srv881025514:84 , srv893870260:84 , srv921749927:84 , srv936136186:84 , srv936567861:84 , srv937404523:84 , srv964894070:84 , srv969330723:84 , srv980377636:84 , srv9929610:84 , srv560837146:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-03 16:53:14,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-03 16:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2069188445=47, srv527467538=66, srv769930033=82, srv937404523=95, srv540419818=68, srv526000162=65, srv616830885=74, srv1094165374=6, srv1740300804=34, srv20643178=46, srv758171196=81, srv666167297=77, srv156245486=22, srv175862891=37, srv234430425=52, srv1580823119=23, srv216155200=48, srv632303836=75, srv1703373458=28, srv397499602=60, srv866517197=86, srv536973908=67, srv730190265=80, srv1228756369=10, srv788291095=84, srv879533808=89, srv335728627=59, srv1718288072=32, srv1881642444=40, srv1673696929=26, srv1706637487=30, srv67188395=78, srv1517659689=21, srv1117409009=8, srv2027142322=45, srv545591806=69, srv1372707832=18, srv497203538=62, srv980377636=98, srv893870260=91, srv225828617=50, srv1581182755=24, srv649481986=76, srv1740831814=35, srv294063743=57, srv560837146=70, srv936567861=94, srv126571553=12, srv105009291=2, srv1018486873=1, srv693463839=79, srv921749927=92, srv2012081606=44, srv1065162553=3, srv216380191=49, srv243688819=53, srv1018108581=0, srv274902087=56, srv1472926160=20, srv1992594773=42, srv1842719227=38, srv881025514=90, srv9929610=99, srv600914303=73, srv1291020759=14, srv594415757=72, srv869465730=87, srv1994950894=43, srv1747292536=36, srv858883599=85, srv1312578900=15, srv871874184=88, srv272361757=55, srv440139438=61, srv1281645432=13, srv1912877484=41, srv1083352573=5, srv591992603=71, srv501439699=63, srv262732033=54, srv1705673046=29, srv511253068=64, srv1698858602=27, srv936136186=93, srv1353623804=17, srv1352235037=16, srv1628153859=25, srv1723460186=33, srv323430857=58, srv1109545094=7, srv1850187373=39, srv969330723=97, srv1457184385=19, srv1179666073=9, srv233439362=51, srv782667795=83, srv1232465744=11, srv1065478879=4, srv964894070=96, srv1715519832=31} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-03 16:53:15,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-03 16:53:15,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-03 16:53:15,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-03 16:53:15,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-03 16:53:15,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-03 16:53:15,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-03 16:53:15,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-03 16:53:15,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-03 16:53:15,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-03 16:53:15,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-03 16:53:15,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-03 16:53:15,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-03 16:53:15,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-03 16:53:15,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-03 16:53:15,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-03 16:53:15,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-03 16:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-03 16:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-03 16:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-03 16:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-03 16:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-03 16:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-03 16:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-03 16:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-03 16:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-03 16:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-03 16:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-03 16:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-03 16:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-03 16:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-03 16:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-03 16:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-03 16:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-03 16:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-03 16:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-03 16:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-03 16:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-03 16:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-03 16:53:15,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-03 16:53:15,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-03 16:53:15,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-03 16:53:15,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-03 16:53:15,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-03 16:53:15,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-03 16:53:15,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-03 16:53:15,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-03 16:53:15,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-03 16:53:15,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-03 16:53:15,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-03 16:53:15,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-03 16:53:15,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-03 16:53:15,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-03 16:53:15,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-03 16:53:15,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-03 16:53:15,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-03 16:53:15,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-03 16:53:15,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-03 16:53:15,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-03 16:53:15,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-03 16:53:15,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-03 16:53:15,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-03 16:53:15,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-03 16:53:15,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-03 16:53:15,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-03 16:53:15,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-03 16:53:15,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-03 16:53:15,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-03 16:53:15,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-03 16:53:15,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-03 16:53:15,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-03 16:53:15,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-03 16:53:15,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-03 16:53:15,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-03 16:53:15,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-03 16:53:15,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-03 16:53:15,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-03 16:53:15,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-03 16:53:15,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-03 16:53:15,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-03 16:53:15,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-03 16:53:15,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-03 16:53:15,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-03 16:53:15,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-03 16:53:15,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-03 16:53:15,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-03 16:53:15,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-03 16:53:15,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-03 16:53:15,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-03 16:53:15,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-03 16:53:15,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-03 16:53:15,465 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:15,465 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-03 16:53:15,466 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-03 16:53:15,834 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 729 ms to try 5000 different iterations.  Found a solution that moves 4713 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18001405687481295. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03770409398301708, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04983474373869783, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5236666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03770409398301708, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17475, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24716666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:15,847 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv869465730:81 , srv1472926160:82 , srv969330723:82 , srv1065162553:83 , srv1083352573:83 , srv1109545094:83 , srv1281645432:83 , srv1353623804:83 , srv1718288072:83 , srv175862891:83 , srv397499602:83 , srv649481986:83 , srv126571553:84 , srv1457184385:84 , srv1703373458:84 , srv294063743:84 , srv526000162:84 , srv632303836:84 , srv871874184:84 , srv1018108581:85 , srv1232465744:85 , srv1312578900:85 , srv1372707832:85 , srv1723460186:85 , srv1992594773:85 , srv20643178:85 , srv234430425:85 , srv243688819:85 , srv272361757:85 , srv440139438:85 , srv545591806:85 , srv600914303:85 , srv866517197:85 , srv879533808:85 , srv936136186:85 , srv937404523:85 , srv1018486873:86 , srv1581182755:86 , srv1628153859:86 , srv1698858602:86 , srv1706637487:86 , srv1881642444:86 , srv616830885:86 , srv693463839:86 , srv730190265:86 , srv782667795:86 , srv788291095:86 , srv881025514:86 , srv893870260:86 , srv921749927:86 , srv105009291:87 , srv1580823119:87 , srv1673696929:87 , srv1740300804:87 , srv1740831814:87 , srv1842719227:87 , srv1994950894:87 , srv216155200:87 , srv225828617:87 , srv233439362:87 , srv274902087:87 , srv501439699:87 , srv666167297:87 , srv980377636:87 , srv1065478879:88 , srv1094165374:88 , srv1179666073:88 , srv1291020759:88 , srv1352235037:88 , srv2027142322:88 , srv2069188445:88 , srv216380191:88 , srv511253068:88 , srv858883599:88 , srv964894070:88 , srv1117409009:89 , srv1228756369:89 , srv156245486:89 , srv1705673046:89 , srv1747292536:89 , srv1912877484:89 , srv323430857:89 , srv335728627:89 , srv536973908:89 , srv540419818:89 , srv591992603:89 , srv67188395:89 , srv9929610:89 , srv1715519832:90 , srv1850187373:90 , srv2012081606:90 , srv527467538:90 , srv758171196:90 , srv936567861:90 , srv594415757:91 , srv1517659689:92 , srv497203538:93 , srv769930033:93 , srv262732033:94 , srv560837146:425 }
2023-08-03 16:53:15,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-03 16:53:15,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2069188445=47, srv527467538=66, srv769930033=82, srv937404523=95, srv540419818=68, srv526000162=65, srv616830885=74, srv1094165374=6, srv1740300804=34, srv20643178=46, srv758171196=81, srv666167297=77, srv156245486=22, srv175862891=37, srv234430425=52, srv1580823119=23, srv216155200=48, srv632303836=75, srv1703373458=28, srv397499602=60, srv866517197=86, srv536973908=67, srv730190265=80, srv1228756369=10, srv788291095=84, srv879533808=89, srv335728627=59, srv1718288072=32, srv1881642444=40, srv1673696929=26, srv1706637487=30, srv67188395=78, srv1517659689=21, srv1117409009=8, srv2027142322=45, srv545591806=69, srv1372707832=18, srv497203538=62, srv980377636=98, srv893870260=91, srv225828617=50, srv1581182755=24, srv649481986=76, srv1740831814=35, srv294063743=57, srv560837146=70, srv936567861=94, srv126571553=12, srv105009291=2, srv1018486873=1, srv693463839=79, srv921749927=92, srv2012081606=44, srv1065162553=3, srv216380191=49, srv243688819=53, srv1018108581=0, srv274902087=56, srv1472926160=20, srv1992594773=42, srv1842719227=38, srv881025514=90, srv9929610=99, srv600914303=73, srv1291020759=14, srv594415757=72, srv869465730=87, srv1994950894=43, srv1747292536=36, srv858883599=85, srv1312578900=15, srv871874184=88, srv272361757=55, srv440139438=61, srv1281645432=13, srv1912877484=41, srv1083352573=5, srv591992603=71, srv501439699=63, srv262732033=54, srv1705673046=29, srv511253068=64, srv1698858602=27, srv936136186=93, srv1353623804=17, srv1352235037=16, srv1628153859=25, srv1723460186=33, srv323430857=58, srv1109545094=7, srv1850187373=39, srv969330723=97, srv1457184385=19, srv1179666073=9, srv233439362=51, srv782667795=83, srv1232465744=11, srv1065478879=4, srv964894070=96, srv1715519832=31} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-03 16:53:15,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-03 16:53:15,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-03 16:53:15,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-03 16:53:15,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-03 16:53:15,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-03 16:53:15,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-03 16:53:15,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-03 16:53:15,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-03 16:53:15,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-03 16:53:15,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-03 16:53:15,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-03 16:53:15,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-03 16:53:15,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-03 16:53:15,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-03 16:53:15,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-03 16:53:15,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-03 16:53:15,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-03 16:53:15,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-03 16:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-03 16:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-03 16:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-03 16:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-03 16:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-03 16:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-03 16:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-03 16:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-03 16:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-03 16:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-03 16:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-03 16:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-03 16:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-03 16:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-03 16:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-03 16:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-03 16:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-03 16:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-03 16:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-03 16:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-03 16:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-03 16:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-03 16:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-03 16:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-03 16:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-03 16:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-03 16:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-03 16:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-03 16:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-03 16:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-03 16:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-03 16:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-03 16:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-03 16:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-03 16:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-03 16:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-03 16:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-03 16:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-03 16:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-03 16:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-03 16:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-03 16:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-03 16:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-03 16:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-03 16:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-03 16:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-03 16:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-03 16:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-03 16:53:15,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-03 16:53:15,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-03 16:53:15,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-03 16:53:15,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-03 16:53:15,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-03 16:53:15,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-03 16:53:15,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-03 16:53:15,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-03 16:53:15,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-03 16:53:15,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-03 16:53:15,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-03 16:53:15,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-03 16:53:15,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-03 16:53:15,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-03 16:53:15,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-03 16:53:15,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-03 16:53:15,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-03 16:53:15,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-03 16:53:15,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-03 16:53:15,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-03 16:53:15,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-03 16:53:15,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-03 16:53:15,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-03 16:53:15,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-03 16:53:15,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-03 16:53:15,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-03 16:53:15,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-03 16:53:15,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-03 16:53:15,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-03 16:53:15,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-03 16:53:15,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-03 16:53:15,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-03 16:53:15,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-03 16:53:15,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-03 16:53:15,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-03 16:53:16,122 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03770409398301708, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04983474373869783, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03770409398301708, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17475, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24716666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:16,122 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-03 16:53:16,123 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17998105574346562, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03770409398301708, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04983474373869783, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03770409398301708, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17475, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24716666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-03 16:53:16,446 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 533 ms to try 5000 different iterations.  Found a solution that moves 2641 regions; Going from a computed imbalance of 0.17998105574346562 to a new imbalance of 0.015473216680259647. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020592907489780397); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02796101181678127, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29344444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020592907489780397); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:16,457 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1517659689:79 , srv262732033:79 , srv497203538:79 , srv527467538:80 , srv769930033:80 , srv536973908:81 , srv936567861:81 , srv9929610:81 , srv1291020759:82 , srv1747292536:82 , srv1850187373:82 , srv2027142322:82 , srv216155200:82 , srv540419818:82 , srv67188395:82 , srv1179666073:83 , srv323430857:83 , srv591992603:83 , srv594415757:83 , srv858883599:83 , srv1117409009:84 , srv1228756369:84 , srv1312578900:84 , srv1352235037:84 , srv156245486:84 , srv1580823119:84 , srv1705673046:84 , srv1706637487:84 , srv1715519832:84 , srv1912877484:84 , srv2012081606:84 , srv216380191:84 , srv233439362:84 , srv335728627:84 , srv440139438:84 , srv758171196:84 , srv1065478879:85 , srv1740300804:85 , srv1994950894:85 , srv20643178:85 , srv2069188445:85 , srv225828617:85 , srv274902087:85 , srv616830885:85 , srv666167297:85 , srv730190265:85 , srv782667795:85 , srv921749927:85 , srv964894070:85 , srv980377636:85 , srv105009291:86 , srv1094165374:86 , srv1372707832:86 , srv1628153859:86 , srv1698858602:86 , srv1723460186:86 , srv1740831814:86 , srv501439699:86 , srv511253068:86 , srv936136186:86 , srv937404523:86 , srv1018486873:87 , srv1581182755:87 , srv1703373458:87 , srv1842719227:87 , srv1992594773:87 , srv234430425:87 , srv243688819:87 , srv272361757:87 , srv294063743:87 , srv693463839:87 , srv866517197:87 , srv871874184:87 , srv881025514:87 , srv1083352573:88 , srv1232465744:88 , srv1457184385:88 , srv1673696929:88 , srv1881642444:88 , srv649481986:88 , srv788291095:88 , srv879533808:88 , srv893870260:88 , srv1018108581:89 , srv1065162553:89 , srv126571553:89 , srv1281645432:89 , srv1353623804:89 , srv1718288072:89 , srv526000162:89 , srv545591806:89 , srv600914303:89 , srv632303836:89 , srv969330723:89 , srv175862891:90 , srv1109545094:91 , srv1472926160:91 , srv397499602:91 , srv869465730:94 , srv560837146:532 }
2023-08-03 16:53:16,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-03 16:53:16,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2069188445=47, srv527467538=66, srv769930033=82, srv937404523=95, srv540419818=68, srv526000162=65, srv616830885=74, srv1094165374=6, srv1740300804=34, srv20643178=46, srv758171196=81, srv666167297=77, srv156245486=22, srv175862891=37, srv234430425=52, srv1580823119=23, srv216155200=48, srv632303836=75, srv1703373458=28, srv397499602=60, srv866517197=86, srv536973908=67, srv730190265=80, srv1228756369=10, srv788291095=84, srv879533808=89, srv335728627=59, srv1718288072=32, srv1881642444=40, srv1673696929=26, srv1706637487=30, srv67188395=78, srv1517659689=21, srv1117409009=8, srv2027142322=45, srv545591806=69, srv1372707832=18, srv497203538=62, srv980377636=98, srv893870260=91, srv225828617=50, srv1581182755=24, srv649481986=76, srv1740831814=35, srv294063743=57, srv560837146=70, srv936567861=94, srv126571553=12, srv105009291=2, srv1018486873=1, srv693463839=79, srv921749927=92, srv2012081606=44, srv1065162553=3, srv216380191=49, srv243688819=53, srv1018108581=0, srv274902087=56, srv1472926160=20, srv1992594773=42, srv1842719227=38, srv881025514=90, srv9929610=99, srv600914303=73, srv1291020759=14, srv594415757=72, srv869465730=87, srv1994950894=43, srv1747292536=36, srv858883599=85, srv1312578900=15, srv871874184=88, srv272361757=55, srv440139438=61, srv1281645432=13, srv1912877484=41, srv1083352573=5, srv591992603=71, srv501439699=63, srv262732033=54, srv1705673046=29, srv511253068=64, srv1698858602=27, srv936136186=93, srv1353623804=17, srv1352235037=16, srv1628153859=25, srv1723460186=33, srv323430857=58, srv1109545094=7, srv1850187373=39, srv969330723=97, srv1457184385=19, srv1179666073=9, srv233439362=51, srv782667795=83, srv1232465744=11, srv1065478879=4, srv964894070=96, srv1715519832=31} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-03 16:53:16,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-03 16:53:16,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-03 16:53:16,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-03 16:53:16,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-03 16:53:16,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-03 16:53:16,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-03 16:53:16,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-03 16:53:16,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-03 16:53:16,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-03 16:53:16,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-03 16:53:16,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-03 16:53:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-03 16:53:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-03 16:53:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-03 16:53:16,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-03 16:53:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-03 16:53:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-03 16:53:16,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-03 16:53:16,705 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020592907489780397); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02796101181678127, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020592907489780397); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:16,705 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-03 16:53:16,705 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015454724003007729, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020592907489780397); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02796101181678127, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020592907489780397); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-03 16:53:16,946 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 413 ms to try 5000 different iterations.  Found a solution that moves 988 regions; Going from a computed imbalance of 0.015454724003007729 to a new imbalance of 4.472447095361243E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0047535601087442675); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006067336330376245); MoveCostFunction : (multiplier=7.0, imbalance=0.10977777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0047535601087442675); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:16,948 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1673696929:82 , srv869465730:82 , srv1094165374:83 , srv1715519832:83 , srv1842719227:83 , srv1881642444:83 , srv2012081606:83 , srv511253068:83 , srv545591806:83 , srv788291095:83 , srv1018108581:84 , srv1065478879:84 , srv1109545094:84 , srv1117409009:84 , srv1228756369:84 , srv1353623804:84 , srv1581182755:84 , srv2069188445:84 , srv262732033:84 , srv335728627:84 , srv397499602:84 , srv501439699:84 , srv594415757:84 , srv600914303:84 , srv758171196:84 , srv879533808:84 , srv893870260:84 , srv1018486873:85 , srv105009291:85 , srv1232465744:85 , srv126571553:85 , srv1352235037:85 , srv1457184385:85 , srv1472926160:85 , srv156245486:85 , srv1628153859:85 , srv1698858602:85 , srv1705673046:85 , srv175862891:85 , srv1912877484:85 , srv1992594773:85 , srv1994950894:85 , srv274902087:85 , srv526000162:85 , srv632303836:85 , srv693463839:85 , srv769930033:85 , srv881025514:85 , srv964894070:85 , srv1065162553:86 , srv1179666073:86 , srv1281645432:86 , srv1372707832:86 , srv1580823119:86 , srv1718288072:86 , srv1740300804:86 , srv1740831814:86 , srv1747292536:86 , srv1850187373:86 , srv216380191:86 , srv225828617:86 , srv234430425:86 , srv243688819:86 , srv272361757:86 , srv294063743:86 , srv323430857:86 , srv591992603:86 , srv666167297:86 , srv730190265:86 , srv866517197:86 , srv921749927:86 , srv937404523:86 , srv969330723:86 , srv980377636:86 , srv1083352573:87 , srv1517659689:87 , srv1703373458:87 , srv1706637487:87 , srv1723460186:87 , srv20643178:87 , srv233439362:87 , srv497203538:87 , srv527467538:87 , srv536973908:87 , srv540419818:87 , srv616830885:87 , srv649481986:87 , srv67188395:87 , srv782667795:87 , srv858883599:87 , srv871874184:87 , srv936136186:87 , srv936567861:87 , srv1291020759:88 , srv1312578900:88 , srv2027142322:88 , srv216155200:88 , srv440139438:88 , srv9929610:88 , srv560837146:543 }
2023-08-03 16:53:16,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-03 16:53:16,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2069188445=47, srv527467538=66, srv769930033=82, srv937404523=95, srv540419818=68, srv526000162=65, srv616830885=74, srv1094165374=6, srv1740300804=34, srv20643178=46, srv758171196=81, srv666167297=77, srv156245486=22, srv175862891=37, srv234430425=52, srv1580823119=23, srv216155200=48, srv632303836=75, srv1703373458=28, srv397499602=60, srv866517197=86, srv536973908=67, srv730190265=80, srv1228756369=10, srv788291095=84, srv879533808=89, srv335728627=59, srv1718288072=32, srv1881642444=40, srv1673696929=26, srv1706637487=30, srv67188395=78, srv1517659689=21, srv1117409009=8, srv2027142322=45, srv545591806=69, srv1372707832=18, srv497203538=62, srv980377636=98, srv893870260=91, srv225828617=50, srv1581182755=24, srv649481986=76, srv1740831814=35, srv294063743=57, srv560837146=70, srv936567861=94, srv126571553=12, srv105009291=2, srv1018486873=1, srv693463839=79, srv921749927=92, srv2012081606=44, srv1065162553=3, srv216380191=49, srv243688819=53, srv1018108581=0, srv274902087=56, srv1472926160=20, srv1992594773=42, srv1842719227=38, srv881025514=90, srv9929610=99, srv600914303=73, srv1291020759=14, srv594415757=72, srv869465730=87, srv1994950894=43, srv1747292536=36, srv858883599=85, srv1312578900=15, srv871874184=88, srv272361757=55, srv440139438=61, srv1281645432=13, srv1912877484=41, srv1083352573=5, srv591992603=71, srv501439699=63, srv262732033=54, srv1705673046=29, srv511253068=64, srv1698858602=27, srv936136186=93, srv1353623804=17, srv1352235037=16, srv1628153859=25, srv1723460186=33, srv323430857=58, srv1109545094=7, srv1850187373=39, srv969330723=97, srv1457184385=19, srv1179666073=9, srv233439362=51, srv782667795=83, srv1232465744=11, srv1065478879=4, srv964894070=96, srv1715519832=31} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-03 16:53:16,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-03 16:53:16,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-03 16:53:16,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-03 16:53:16,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-03 16:53:16,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-03 16:53:16,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-03 16:53:16,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-03 16:53:16,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-03 16:53:16,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-03 16:53:16,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-03 16:53:16,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-03 16:53:16,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-03 16:53:16,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-03 16:53:16,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-03 16:53:16,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-03 16:53:16,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-03 16:53:16,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-03 16:53:16,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-03 16:53:16,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-03 16:53:16,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-03 16:53:16,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-03 16:53:16,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-03 16:53:16,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-03 16:53:16,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-03 16:53:16,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-03 16:53:16,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-03 16:53:16,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-03 16:53:16,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-03 16:53:16,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-03 16:53:16,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-03 16:53:16,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-03 16:53:16,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-03 16:53:16,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-03 16:53:16,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-03 16:53:16,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-03 16:53:16,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-03 16:53:16,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-03 16:53:16,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-03 16:53:16,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-03 16:53:16,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-03 16:53:16,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-03 16:53:16,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-03 16:53:16,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-03 16:53:16,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-03 16:53:16,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-03 16:53:16,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-03 16:53:16,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-03 16:53:16,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-03 16:53:16,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-03 16:53:16,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-03 16:53:16,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-03 16:53:16,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-03 16:53:16,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-03 16:53:16,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-03 16:53:16,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-03 16:53:16,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-03 16:53:16,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-03 16:53:16,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-03 16:53:16,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-03 16:53:16,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-03 16:53:16,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-03 16:53:16,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-03 16:53:16,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-03 16:53:16,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-03 16:53:16,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-03 16:53:16,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-03 16:53:16,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-03 16:53:16,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-03 16:53:16,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-03 16:53:16,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-03 16:53:16,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-03 16:53:16,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-03 16:53:16,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-03 16:53:16,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-03 16:53:16,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-03 16:53:16,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-03 16:53:16,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-03 16:53:16,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-03 16:53:16,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-03 16:53:16,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-03 16:53:16,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-03 16:53:16,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-03 16:53:16,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-03 16:53:16,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-03 16:53:16,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-03 16:53:16,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-03 16:53:16,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-03 16:53:16,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-03 16:53:16,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-03 16:53:16,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-03 16:53:16,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-03 16:53:16,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-03 16:53:16,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-03 16:53:16,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-03 16:53:16,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-03 16:53:16,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-03 16:53:16,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-03 16:53:16,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-03 16:53:16,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-03 16:53:16,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-03 16:53:16,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-03 16:53:17,138 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0047535601087442675); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006067336330376245); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0047535601087442675); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:17,138 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-03 16:53:17,142 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.403265856720981E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0047535601087442675); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006067336330376245); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0047535601087442675); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-03 16:53:17,649 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 646 ms to try 5000 different iterations.  Found a solution that moves 205 regions; Going from a computed imbalance of 4.403265856720981E-4 to a new imbalance of 2.657117862866837E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.02277777777777778); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:17,651 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv105009291:83 , srv1703373458:83 , srv1740831814:83 , srv497203538:83 , srv1018108581:84 , srv1018486873:84 , srv1083352573:84 , srv1109545094:84 , srv1232465744:84 , srv126571553:84 , srv1281645432:84 , srv1291020759:84 , srv1472926160:84 , srv156245486:84 , srv1705673046:84 , srv1706637487:84 , srv1718288072:84 , srv1740300804:84 , srv175862891:84 , srv1850187373:84 , srv1912877484:84 , srv1992594773:84 , srv2027142322:84 , srv20643178:84 , srv2069188445:84 , srv216380191:84 , srv225828617:84 , srv234430425:84 , srv243688819:84 , srv272361757:84 , srv323430857:84 , srv397499602:84 , srv527467538:84 , srv540419818:84 , srv594415757:84 , srv600914303:84 , srv616830885:84 , srv632303836:84 , srv649481986:84 , srv666167297:84 , srv67188395:84 , srv693463839:84 , srv730190265:84 , srv758171196:84 , srv769930033:84 , srv782667795:84 , srv858883599:84 , srv866517197:84 , srv871874184:84 , srv881025514:84 , srv893870260:84 , srv936136186:84 , srv936567861:84 , srv964894070:84 , srv980377636:84 , srv9929610:84 , srv1065162553:85 , srv1065478879:85 , srv1094165374:85 , srv1117409009:85 , srv1179666073:85 , srv1228756369:85 , srv1312578900:85 , srv1352235037:85 , srv1372707832:85 , srv1457184385:85 , srv1517659689:85 , srv1580823119:85 , srv1581182755:85 , srv1628153859:85 , srv1673696929:85 , srv1698858602:85 , srv1715519832:85 , srv1723460186:85 , srv1747292536:85 , srv1842719227:85 , srv1881642444:85 , srv1994950894:85 , srv2012081606:85 , srv216155200:85 , srv233439362:85 , srv262732033:85 , srv274902087:85 , srv294063743:85 , srv335728627:85 , srv440139438:85 , srv501439699:85 , srv511253068:85 , srv526000162:85 , srv536973908:85 , srv545591806:85 , srv591992603:85 , srv788291095:85 , srv869465730:85 , srv921749927:85 , srv937404523:85 , srv969330723:85 , srv1353623804:86 , srv879533808:86 , srv560837146:643 }
2023-08-03 16:53:17,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-03 16:53:17,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2069188445=47, srv527467538=66, srv769930033=82, srv937404523=95, srv540419818=68, srv526000162=65, srv616830885=74, srv1094165374=6, srv1740300804=34, srv20643178=46, srv758171196=81, srv666167297=77, srv156245486=22, srv175862891=37, srv234430425=52, srv1580823119=23, srv216155200=48, srv632303836=75, srv1703373458=28, srv397499602=60, srv866517197=86, srv536973908=67, srv730190265=80, srv1228756369=10, srv788291095=84, srv879533808=89, srv335728627=59, srv1718288072=32, srv1881642444=40, srv1673696929=26, srv1706637487=30, srv67188395=78, srv1517659689=21, srv1117409009=8, srv2027142322=45, srv545591806=69, srv1372707832=18, srv497203538=62, srv980377636=98, srv893870260=91, srv225828617=50, srv1581182755=24, srv649481986=76, srv1740831814=35, srv294063743=57, srv560837146=70, srv936567861=94, srv126571553=12, srv105009291=2, srv1018486873=1, srv693463839=79, srv921749927=92, srv2012081606=44, srv1065162553=3, srv216380191=49, srv243688819=53, srv1018108581=0, srv274902087=56, srv1472926160=20, srv1992594773=42, srv1842719227=38, srv881025514=90, srv9929610=99, srv600914303=73, srv1291020759=14, srv594415757=72, srv869465730=87, srv1994950894=43, srv1747292536=36, srv858883599=85, srv1312578900=15, srv871874184=88, srv272361757=55, srv440139438=61, srv1281645432=13, srv1912877484=41, srv1083352573=5, srv591992603=71, srv501439699=63, srv262732033=54, srv1705673046=29, srv511253068=64, srv1698858602=27, srv936136186=93, srv1353623804=17, srv1352235037=16, srv1628153859=25, srv1723460186=33, srv323430857=58, srv1109545094=7, srv1850187373=39, srv969330723=97, srv1457184385=19, srv1179666073=9, srv233439362=51, srv782667795=83, srv1232465744=11, srv1065478879=4, srv964894070=96, srv1715519832=31} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-03 16:53:17,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-03 16:53:17,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-03 16:53:17,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-03 16:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-03 16:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-03 16:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-03 16:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-03 16:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-03 16:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-03 16:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-03 16:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-03 16:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-03 16:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-03 16:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-03 16:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-03 16:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-03 16:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-03 16:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-03 16:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-03 16:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-03 16:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-03 16:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-03 16:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-03 16:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-03 16:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-03 16:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-03 16:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-03 16:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-03 16:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-03 16:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-03 16:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-03 16:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-03 16:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-03 16:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-03 16:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-03 16:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-03 16:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-03 16:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-03 16:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-03 16:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-03 16:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-03 16:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-03 16:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-03 16:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-03 16:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-03 16:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-03 16:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-03 16:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-03 16:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-03 16:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-03 16:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-03 16:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-03 16:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-03 16:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-03 16:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-03 16:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-03 16:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-03 16:53:17,824 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:17,824 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-03 16:53:17,825 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.513573794837952E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-03 16:53:18,105 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 409 ms to try 5000 different iterations.  Found a solution that moves 20 regions; Going from a computed imbalance of 2.513573794837952E-5 to a new imbalance of 6.88411204703086E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0022222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:18,106 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1065162553:83 , srv1517659689:83 , srv1723460186:83 , srv233439362:83 , srv526000162:83 , srv591992603:83 , srv879533808:83 , srv1018108581:84 , srv1018486873:84 , srv1065478879:84 , srv1083352573:84 , srv1094165374:84 , srv1109545094:84 , srv1117409009:84 , srv1179666073:84 , srv1228756369:84 , srv1232465744:84 , srv126571553:84 , srv1281645432:84 , srv1291020759:84 , srv1312578900:84 , srv1352235037:84 , srv1353623804:84 , srv1372707832:84 , srv1457184385:84 , srv1472926160:84 , srv156245486:84 , srv1580823119:84 , srv1581182755:84 , srv1628153859:84 , srv1673696929:84 , srv1698858602:84 , srv1705673046:84 , srv1715519832:84 , srv1718288072:84 , srv1740300804:84 , srv1740831814:84 , srv1747292536:84 , srv175862891:84 , srv1842719227:84 , srv1850187373:84 , srv1881642444:84 , srv1912877484:84 , srv1994950894:84 , srv2012081606:84 , srv2027142322:84 , srv216155200:84 , srv216380191:84 , srv225828617:84 , srv234430425:84 , srv243688819:84 , srv262732033:84 , srv272361757:84 , srv274902087:84 , srv294063743:84 , srv323430857:84 , srv335728627:84 , srv397499602:84 , srv440139438:84 , srv497203538:84 , srv501439699:84 , srv511253068:84 , srv536973908:84 , srv540419818:84 , srv545591806:84 , srv594415757:84 , srv600914303:84 , srv616830885:84 , srv632303836:84 , srv649481986:84 , srv666167297:84 , srv67188395:84 , srv693463839:84 , srv758171196:84 , srv769930033:84 , srv782667795:84 , srv788291095:84 , srv858883599:84 , srv866517197:84 , srv869465730:84 , srv871874184:84 , srv881025514:84 , srv893870260:84 , srv921749927:84 , srv936136186:84 , srv936567861:84 , srv937404523:84 , srv964894070:84 , srv969330723:84 , srv980377636:84 , srv9929610:84 , srv105009291:85 , srv1703373458:85 , srv1706637487:85 , srv1992594773:85 , srv20643178:85 , srv2069188445:85 , srv527467538:85 , srv730190265:85 , srv560837146:683 }
2023-08-03 16:53:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-03 16:53:18,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2069188445=47, srv527467538=66, srv769930033=82, srv937404523=95, srv540419818=68, srv526000162=65, srv616830885=74, srv1094165374=6, srv1740300804=34, srv20643178=46, srv758171196=81, srv666167297=77, srv156245486=22, srv175862891=37, srv234430425=52, srv1580823119=23, srv216155200=48, srv632303836=75, srv1703373458=28, srv397499602=60, srv866517197=86, srv536973908=67, srv730190265=80, srv1228756369=10, srv788291095=84, srv879533808=89, srv335728627=59, srv1718288072=32, srv1881642444=40, srv1673696929=26, srv1706637487=30, srv67188395=78, srv1517659689=21, srv1117409009=8, srv2027142322=45, srv545591806=69, srv1372707832=18, srv497203538=62, srv980377636=98, srv893870260=91, srv225828617=50, srv1581182755=24, srv649481986=76, srv1740831814=35, srv294063743=57, srv560837146=70, srv936567861=94, srv126571553=12, srv105009291=2, srv1018486873=1, srv693463839=79, srv921749927=92, srv2012081606=44, srv1065162553=3, srv216380191=49, srv243688819=53, srv1018108581=0, srv274902087=56, srv1472926160=20, srv1992594773=42, srv1842719227=38, srv881025514=90, srv9929610=99, srv600914303=73, srv1291020759=14, srv594415757=72, srv869465730=87, srv1994950894=43, srv1747292536=36, srv858883599=85, srv1312578900=15, srv871874184=88, srv272361757=55, srv440139438=61, srv1281645432=13, srv1912877484=41, srv1083352573=5, srv591992603=71, srv501439699=63, srv262732033=54, srv1705673046=29, srv511253068=64, srv1698858602=27, srv936136186=93, srv1353623804=17, srv1352235037=16, srv1628153859=25, srv1723460186=33, srv323430857=58, srv1109545094=7, srv1850187373=39, srv969330723=97, srv1457184385=19, srv1179666073=9, srv233439362=51, srv782667795=83, srv1232465744=11, srv1065478879=4, srv964894070=96, srv1715519832=31} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-03 16:53:18,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-03 16:53:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-03 16:53:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-03 16:53:18,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-03 16:53:18,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-03 16:53:18,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-03 16:53:18,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-03 16:53:18,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-03 16:53:18,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-03 16:53:18,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-03 16:53:18,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-03 16:53:18,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-03 16:53:18,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-03 16:53:18,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-03 16:53:18,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-03 16:53:18,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-03 16:53:18,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-03 16:53:18,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-03 16:53:18,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-03 16:53:18,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-03 16:53:18,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-03 16:53:18,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-03 16:53:18,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-03 16:53:18,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-03 16:53:18,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-03 16:53:18,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-03 16:53:18,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-03 16:53:18,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-03 16:53:18,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-03 16:53:18,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-03 16:53:18,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-03 16:53:18,272 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:18,272 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1065162553:83 , srv1517659689:83 , srv1723460186:83 , srv233439362:83 , srv526000162:83 , srv591992603:83 , srv879533808:83 , srv1018108581:84 , srv1018486873:84 , srv1065478879:84 , srv1083352573:84 , srv1094165374:84 , srv1109545094:84 , srv1117409009:84 , srv1179666073:84 , srv1228756369:84 , srv1232465744:84 , srv126571553:84 , srv1281645432:84 , srv1291020759:84 , srv1312578900:84 , srv1352235037:84 , srv1353623804:84 , srv1372707832:84 , srv1457184385:84 , srv1472926160:84 , srv156245486:84 , srv1580823119:84 , srv1581182755:84 , srv1628153859:84 , srv1673696929:84 , srv1698858602:84 , srv1705673046:84 , srv1715519832:84 , srv1718288072:84 , srv1740300804:84 , srv1740831814:84 , srv1747292536:84 , srv175862891:84 , srv1842719227:84 , srv1850187373:84 , srv1881642444:84 , srv1912877484:84 , srv1994950894:84 , srv2012081606:84 , srv2027142322:84 , srv216155200:84 , srv216380191:84 , srv225828617:84 , srv234430425:84 , srv243688819:84 , srv262732033:84 , srv272361757:84 , srv274902087:84 , srv294063743:84 , srv323430857:84 , srv335728627:84 , srv397499602:84 , srv440139438:84 , srv497203538:84 , srv501439699:84 , srv511253068:84 , srv536973908:84 , srv540419818:84 , srv545591806:84 , srv594415757:84 , srv600914303:84 , srv616830885:84 , srv632303836:84 , srv649481986:84 , srv666167297:84 , srv67188395:84 , srv693463839:84 , srv758171196:84 , srv769930033:84 , srv782667795:84 , srv788291095:84 , srv858883599:84 , srv866517197:84 , srv869465730:84 , srv871874184:84 , srv881025514:84 , srv893870260:84 , srv921749927:84 , srv936136186:84 , srv936567861:84 , srv937404523:84 , srv964894070:84 , srv969330723:84 , srv980377636:84 , srv9929610:84 , srv105009291:85 , srv1703373458:85 , srv1706637487:85 , srv1992594773:85 , srv20643178:85 , srv2069188445:85 , srv527467538:85 , srv730190265:85 , srv560837146:683 }
2023-08-03 16:53:18,304 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=327 (was 294) - SystemLoadAverage LEAK? -, ProcessCount=175 (was 175), AvailableMemoryMB=1758 (was 2640)
2023-08-03 16:53:18,314 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=327, ProcessCount=175, AvailableMemoryMB=1757
2023-08-03 16:53:18,315 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-03 16:53:18,315 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-03 16:53:18,328 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1113346023:84 , srv1145546810:84 , srv1201516940:84 , srv1262899198:84 , srv1308747199:84 , srv1346892141:84 , srv137460042:84 , srv1413387472:84 , srv1483562094:84 , srv1781781104:84 , srv1946549255:84 , srv2026556746:84 , srv2027442042:84 , srv2079390982:84 , srv230875764:84 , srv246581122:84 , srv359054918:84 , srv526272342:84 , srv531239038:84 , srv547936050:84 , srv570968034:84 , srv628995025:84 , srv660049078:84 , srv669228579:84 , srv833687945:84 , srv859030237:84 , srv893290384:84 , srv943301006:84 , srv977012112:84 , srv588497994:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-03 16:53:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-03 16:53:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1145546810=1, srv526272342=17, srv1308747199=4, srv230875764=14, srv893290384=27, srv2027442042=12, srv137460042=6, srv859030237=26, srv2026556746=11, srv1946549255=10, srv547936050=19, srv1413387472=7, srv1262899198=3, srv359054918=16, srv669228579=24, srv1201516940=2, srv570968034=20, srv588497994=21, srv833687945=25, srv1781781104=9, srv628995025=22, srv977012112=29, srv246581122=15, srv1113346023=0, srv531239038=18, srv660049078=23, srv1346892141=5, srv2079390982=13, srv1483562094=8, srv943301006=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-03 16:53:18,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-03 16:53:18,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-03 16:53:18,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-03 16:53:18,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-03 16:53:18,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-03 16:53:18,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-03 16:53:18,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-03 16:53:18,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-03 16:53:18,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-03 16:53:18,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-03 16:53:18,362 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1426380707504196, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:53:18,362 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912642491167653, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1426380707504196, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-03 16:55:18,344 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6110725 different iterations.  Found a solution that moves 1862 regions; Going from a computed imbalance of 0.9912642491167653 to a new imbalance of 4.346000888939571E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6896296296296296, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:55:18,351 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1113346023:90 , srv1145546810:90 , srv1201516940:90 , srv1262899198:90 , srv1308747199:90 , srv1346892141:90 , srv137460042:90 , srv1413387472:90 , srv1483562094:90 , srv1781781104:90 , srv1946549255:90 , srv2026556746:90 , srv2027442042:90 , srv2079390982:90 , srv230875764:90 , srv246581122:90 , srv359054918:90 , srv526272342:90 , srv531239038:90 , srv547936050:90 , srv570968034:90 , srv588497994:90 , srv628995025:90 , srv660049078:90 , srv669228579:90 , srv833687945:90 , srv859030237:90 , srv893290384:90 , srv943301006:90 , srv977012112:90 }
2023-08-03 16:55:18,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-03 16:55:18,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1145546810=1, srv526272342=17, srv1308747199=4, srv230875764=14, srv893290384=27, srv2027442042=12, srv137460042=6, srv859030237=26, srv2026556746=11, srv1946549255=10, srv547936050=19, srv1413387472=7, srv1262899198=3, srv359054918=16, srv669228579=24, srv1201516940=2, srv570968034=20, srv588497994=21, srv833687945=25, srv1781781104=9, srv628995025=22, srv977012112=29, srv246581122=15, srv1113346023=0, srv531239038=18, srv660049078=23, srv1346892141=5, srv2079390982=13, srv1483562094=8, srv943301006=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-03 16:55:18,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-03 16:55:18,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-03 16:55:18,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-03 16:55:18,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-03 16:55:18,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-03 16:55:18,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-03 16:55:18,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-03 16:55:18,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-03 16:55:18,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-03 16:55:18,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-03 16:55:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-03 16:55:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-03 16:55:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-03 16:55:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-03 16:55:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-03 16:55:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-03 16:55:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-03 16:55:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-03 16:55:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-03 16:55:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-03 16:55:18,377 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-03 16:55:18,378 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1113346023:90 , srv1145546810:90 , srv1201516940:90 , srv1262899198:90 , srv1308747199:90 , srv1346892141:90 , srv137460042:90 , srv1413387472:90 , srv1483562094:90 , srv1781781104:90 , srv1946549255:90 , srv2026556746:90 , srv2027442042:90 , srv2079390982:90 , srv230875764:90 , srv246581122:90 , srv359054918:90 , srv526272342:90 , srv531239038:90 , srv547936050:90 , srv570968034:90 , srv588497994:90 , srv628995025:90 , srv660049078:90 , srv669228579:90 , srv833687945:90 , srv859030237:90 , srv893290384:90 , srv943301006:90 , srv977012112:90 }
2023-08-03 16:55:18,396 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=200 (was 327), ProcessCount=173 (was 175), AvailableMemoryMB=1652 (was 1757)