2023-03-01 21:54:09,645 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-01 21:54:10,012 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-01 21:54:10,175 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-01 21:54:10,237 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-01 21:54:10,245 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-01 21:54:10,258 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=360, ProcessCount=183, AvailableMemoryMB=8318
2023-03-01 21:54:10,260 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-01 21:54:10,260 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-01 21:54:10,394 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1014198383:84 , srv1023728695:84 , srv1083397957:84 , srv1090207299:84 , srv113225936:84 , srv1167640182:84 , srv1184754532:84 , srv1210838791:84 , srv1232193747:84 , srv1237150545:84 , srv1245739108:84 , srv1247926960:84 , srv1249087144:84 , srv1293619832:84 , srv1295881204:84 , srv1353680750:84 , srv1398294710:84 , srv1407307746:84 , srv143086438:84 , srv1463017994:84 , srv1465640366:84 , srv1466106279:84 , srv1468416506:84 , srv1474920935:84 , srv148032211:84 , srv1510081425:84 , srv1517347237:84 , srv1533212309:84 , srv1534945258:84 , srv1547616082:84 , srv1552315833:84 , srv1582010656:84 , srv1591640422:84 , srv1607997319:84 , srv1629885558:84 , srv163420653:84 , srv1679067835:84 , srv1695875172:84 , srv1701983192:84 , srv1721379854:84 , srv1727555733:84 , srv1744644383:84 , srv1779411097:84 , srv1829904234:84 , srv1843138429:84 , srv1854167196:84 , srv1915289404:84 , srv1929897788:84 , srv195273696:84 , srv1959889283:84 , srv1965981817:84 , srv1998948936:84 , srv2032789800:84 , srv20690662:84 , srv2126988581:84 , srv2133579536:84 , srv218406726:84 , srv228183504:84 , srv229183957:84 , srv247911853:84 , srv260622554:84 , srv270837987:84 , srv272952275:84 , srv274748606:84 , srv285398425:84 , srv30722910:84 , srv328289272:84 , srv370048868:84 , srv394870176:84 , srv397602587:84 , srv424471837:84 , srv426998906:84 , srv433358318:84 , srv47415013:84 , srv478018047:84 , srv501231148:84 , srv526831715:84 , srv535009071:84 , srv579455823:84 , srv635607770:84 , srv690782150:84 , srv699366386:84 , srv805162654:84 , srv833135100:84 , srv834764472:84 , srv842521404:84 , srv857983825:84 , srv887357996:84 , srv888172191:84 , srv905051589:84 , srv935468523:84 , srv937424392:84 , srv949095833:84 , srv952715834:84 , srv953111222:84 , srv955137432:84 , srv986129162:84 , srv995620768:84 , srv998922009:84 , srv1996877870:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-01 21:54:10,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-01 21:54:10,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148032211=24, srv1090207299=3, srv1465640366=20, srv888172191=89, srv526831715=77, srv1295881204=14, srv1727555733=40, srv986129162=97, srv1249087144=12, srv1510081425=25, srv1023728695=1, srv953111222=95, srv30722910=66, srv1468416506=22, srv328289272=67, srv1293619832=13, srv635607770=80, srv163420653=35, srv1695875172=37, srv690782150=81, srv998922009=99, srv2032789800=53, srv426998906=72, srv1463017994=19, srv1167640182=5, srv1517347237=26, srv1965981817=50, srv143086438=18, srv424471837=71, srv949095833=93, srv1607997319=33, srv247911853=60, srv1014198383=0, srv397602587=70, srv433358318=73, srv955137432=96, srv195273696=48, srv1929897788=47, srv1779411097=42, srv370048868=68, srv1629885558=34, srv834764472=85, srv937424392=92, srv1210838791=7, srv47415013=74, srv995620768=98, srv1843138429=44, srv833135100=84, srv272952275=63, srv218406726=57, srv228183504=58, srv1466106279=21, srv1744644383=41, srv1998948936=52, srv935468523=91, srv1237150545=9, srv1959889283=49, srv905051589=90, srv1591640422=32, srv1854167196=45, srv394870176=69, srv1996877870=51, srv579455823=79, srv1407307746=17, srv1474920935=23, srv229183957=59, srv260622554=61, srv857983825=87, srv113225936=4, srv1679067835=36, srv20690662=54, srv1184754532=6, srv1398294710=16, srv274748606=64, srv1547616082=29, srv1247926960=11, srv1083397957=2, srv842521404=86, srv1915289404=46, srv1552315833=30, srv2126988581=55, srv1534945258=28, srv478018047=75, srv535009071=78, srv1721379854=39, srv2133579536=56, srv805162654=83, srv952715834=94, srv1701983192=38, srv1245739108=10, srv501231148=76, srv270837987=62, srv285398425=65, srv699366386=82, srv1533212309=27, srv887357996=88, srv1353680750=15, srv1582010656=31, srv1232193747=8, srv1829904234=43} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-01 21:54:10,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-01 21:54:10,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-01 21:54:10,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-01 21:54:10,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-01 21:54:10,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-01 21:54:10,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-01 21:54:10,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-01 21:54:10,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-01 21:54:10,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-01 21:54:10,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-01 21:54:10,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-01 21:54:10,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-01 21:54:10,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-01 21:54:10,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-01 21:54:10,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-01 21:54:10,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-01 21:54:10,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-01 21:54:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-01 21:54:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-01 21:54:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-01 21:54:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-01 21:54:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-01 21:54:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-01 21:54:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-01 21:54:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-01 21:54:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-01 21:54:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-01 21:54:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-01 21:54:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-01 21:54:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-01 21:54:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-01 21:54:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-01 21:54:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-01 21:54:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-01 21:54:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-01 21:54:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-01 21:54:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-01 21:54:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-01 21:54:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-01 21:54:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-01 21:54:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-01 21:54:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-01 21:54:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-01 21:54:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-01 21:54:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-01 21:54:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-01 21:54:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-01 21:54:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-01 21:54:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-01 21:54:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-01 21:54:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-01 21:54:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-01 21:54:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-01 21:54:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-01 21:54:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-01 21:54:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-01 21:54:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-01 21:54:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-01 21:54:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-01 21:54:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-01 21:54:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-01 21:54:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-01 21:54:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-01 21:54:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-01 21:54:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-01 21:54:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-01 21:54:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-01 21:54:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-01 21:54:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-01 21:54:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-01 21:54:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-01 21:54:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-01 21:54:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-01 21:54:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-01 21:54:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-01 21:54:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-01 21:54:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-01 21:54:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-01 21:54:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-01 21:54:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-01 21:54:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-01 21:54:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-01 21:54:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-01 21:54:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-01 21:54:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-01 21:54:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-01 21:54:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-01 21:54:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-01 21:54:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-01 21:54:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-01 21:54:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-01 21:54:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-01 21:54:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-01 21:54:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-01 21:54:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-01 21:54:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-01 21:54:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-01 21:54:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-01 21:54:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-01 21:54:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-01 21:54:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-01 21:54:10,771 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:10,772 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-01 21:54:10,772 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-01 21:54:11,133 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 585 ms to try 5000 different iterations.  Found a solution that moves 4668 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18257586722036317. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041560170674662125, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05989515718400379, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5186666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041560170674662125, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17791666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24325, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:11,154 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv397602587:81 , srv949095833:81 , srv1695875172:82 , srv1998948936:82 , srv247911853:82 , srv285398425:82 , srv370048868:82 , srv952715834:82 , srv1245739108:83 , srv1721379854:83 , srv1829904234:83 , srv229183957:83 , srv274748606:83 , srv526831715:83 , srv995620768:83 , srv1293619832:84 , srv148032211:84 , srv1552315833:84 , srv1607997319:84 , srv1843138429:84 , srv2032789800:84 , srv433358318:84 , srv579455823:84 , srv635607770:84 , srv833135100:84 , srv834764472:84 , srv1232193747:85 , srv1398294710:85 , srv1407307746:85 , srv1465640366:85 , srv1474920935:85 , srv1517347237:85 , srv1744644383:85 , srv1854167196:85 , srv1915289404:85 , srv1959889283:85 , srv2133579536:85 , srv30722910:85 , srv478018047:85 , srv501231148:85 , srv857983825:85 , srv935468523:85 , srv998922009:85 , srv1014198383:86 , srv1023728695:86 , srv1167640182:86 , srv1237150545:86 , srv1353680750:86 , srv1510081425:86 , srv1582010656:86 , srv1591640422:86 , srv1779411097:86 , srv1929897788:86 , srv20690662:86 , srv2126988581:86 , srv218406726:86 , srv228183504:86 , srv260622554:86 , srv270837987:86 , srv328289272:86 , srv394870176:86 , srv424471837:86 , srv888172191:86 , srv986129162:86 , srv113225936:87 , srv1249087144:87 , srv1463017994:87 , srv1533212309:87 , srv1629885558:87 , srv195273696:87 , srv1965981817:87 , srv690782150:87 , srv905051589:87 , srv1083397957:88 , srv1184754532:88 , srv1466106279:88 , srv1468416506:88 , srv1534945258:88 , srv1547616082:88 , srv426998906:88 , srv842521404:88 , srv937424392:88 , srv1210838791:89 , srv1295881204:89 , srv163420653:89 , srv1679067835:89 , srv805162654:89 , srv887357996:89 , srv143086438:90 , srv1701983192:90 , srv47415013:90 , srv699366386:90 , srv1727555733:92 , srv535009071:92 , srv953111222:93 , srv955137432:95 , srv1090207299:96 , srv1247926960:96 , srv272952275:98 , srv1996877870:459 }
2023-03-01 21:54:11,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-01 21:54:11,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148032211=24, srv1090207299=3, srv1465640366=20, srv888172191=89, srv526831715=77, srv1295881204=14, srv1727555733=40, srv986129162=97, srv1249087144=12, srv1510081425=25, srv1023728695=1, srv953111222=95, srv30722910=66, srv1468416506=22, srv328289272=67, srv1293619832=13, srv635607770=80, srv163420653=35, srv1695875172=37, srv690782150=81, srv998922009=99, srv2032789800=53, srv426998906=72, srv1463017994=19, srv1167640182=5, srv1517347237=26, srv1965981817=50, srv143086438=18, srv424471837=71, srv949095833=93, srv1607997319=33, srv247911853=60, srv1014198383=0, srv397602587=70, srv433358318=73, srv955137432=96, srv195273696=48, srv1929897788=47, srv1779411097=42, srv370048868=68, srv1629885558=34, srv834764472=85, srv937424392=92, srv1210838791=7, srv47415013=74, srv995620768=98, srv1843138429=44, srv833135100=84, srv272952275=63, srv218406726=57, srv228183504=58, srv1466106279=21, srv1744644383=41, srv1998948936=52, srv935468523=91, srv1237150545=9, srv1959889283=49, srv905051589=90, srv1591640422=32, srv1854167196=45, srv394870176=69, srv1996877870=51, srv579455823=79, srv1407307746=17, srv1474920935=23, srv229183957=59, srv260622554=61, srv857983825=87, srv113225936=4, srv1679067835=36, srv20690662=54, srv1184754532=6, srv1398294710=16, srv274748606=64, srv1547616082=29, srv1247926960=11, srv1083397957=2, srv842521404=86, srv1915289404=46, srv1552315833=30, srv2126988581=55, srv1534945258=28, srv478018047=75, srv535009071=78, srv1721379854=39, srv2133579536=56, srv805162654=83, srv952715834=94, srv1701983192=38, srv1245739108=10, srv501231148=76, srv270837987=62, srv285398425=65, srv699366386=82, srv1533212309=27, srv887357996=88, srv1353680750=15, srv1582010656=31, srv1232193747=8, srv1829904234=43} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-01 21:54:11,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-01 21:54:11,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-01 21:54:11,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-01 21:54:11,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-01 21:54:11,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-01 21:54:11,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-01 21:54:11,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-01 21:54:11,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-01 21:54:11,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-01 21:54:11,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-01 21:54:11,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-01 21:54:11,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-01 21:54:11,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-01 21:54:11,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-01 21:54:11,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-01 21:54:11,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-01 21:54:11,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-01 21:54:11,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-01 21:54:11,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-01 21:54:11,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-01 21:54:11,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-01 21:54:11,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-01 21:54:11,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-01 21:54:11,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-01 21:54:11,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-01 21:54:11,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-01 21:54:11,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-01 21:54:11,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-01 21:54:11,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-01 21:54:11,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-01 21:54:11,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-01 21:54:11,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-01 21:54:11,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-01 21:54:11,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-01 21:54:11,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-01 21:54:11,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-01 21:54:11,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-01 21:54:11,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-01 21:54:11,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-01 21:54:11,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-01 21:54:11,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-01 21:54:11,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-01 21:54:11,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-01 21:54:11,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-01 21:54:11,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-01 21:54:11,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-01 21:54:11,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-01 21:54:11,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-01 21:54:11,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-01 21:54:11,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-01 21:54:11,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-01 21:54:11,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-01 21:54:11,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-01 21:54:11,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-01 21:54:11,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-01 21:54:11,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-01 21:54:11,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-01 21:54:11,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-01 21:54:11,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-01 21:54:11,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-01 21:54:11,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-01 21:54:11,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-01 21:54:11,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-01 21:54:11,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-01 21:54:11,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-01 21:54:11,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-01 21:54:11,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-01 21:54:11,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-01 21:54:11,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-01 21:54:11,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-01 21:54:11,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-01 21:54:11,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-01 21:54:11,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-01 21:54:11,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-01 21:54:11,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-01 21:54:11,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-01 21:54:11,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-01 21:54:11,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-01 21:54:11,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-01 21:54:11,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-01 21:54:11,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-01 21:54:11,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-01 21:54:11,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-01 21:54:11,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-01 21:54:11,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-01 21:54:11,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-01 21:54:11,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-01 21:54:11,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-01 21:54:11,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-01 21:54:11,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-01 21:54:11,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-01 21:54:11,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-01 21:54:11,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-01 21:54:11,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-01 21:54:11,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-01 21:54:11,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-01 21:54:11,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-01 21:54:11,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-01 21:54:11,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-01 21:54:11,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-01 21:54:11,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-01 21:54:11,382 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041560170674662125, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05989515718400379, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041560170674662125, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17791666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24325, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:11,383 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-01 21:54:11,383 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18254318118575055, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041560170674662125, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05989515718400379, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041560170674662125, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17791666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24325, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-01 21:54:11,696 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 486 ms to try 5000 different iterations.  Found a solution that moves 2617 regions; Going from a computed imbalance of 0.18254318118575055 to a new imbalance of 0.017067927774257606. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024531016985917784); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03805601636527022, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2907777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024531016985917784); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01375); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.048666666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:11,714 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1090207299:75 , srv1247926960:75 , srv272952275:76 , srv955137432:77 , srv1727555733:79 , srv1210838791:81 , srv143086438:81 , srv47415013:81 , srv535009071:81 , srv699366386:81 , srv953111222:81 , srv1534945258:82 , srv163420653:82 , srv1679067835:82 , srv1701983192:82 , srv805162654:82 , srv1184754532:83 , srv1249087144:83 , srv1463017994:83 , srv1547616082:83 , srv1629885558:83 , srv1744644383:83 , srv937424392:83 , srv1232193747:84 , srv1466106279:84 , srv1510081425:84 , srv1533212309:84 , srv1779411097:84 , srv2133579536:84 , srv328289272:84 , srv887357996:84 , srv1083397957:85 , srv113225936:85 , srv1295881204:85 , srv1353680750:85 , srv1468416506:85 , srv195273696:85 , srv1965981817:85 , srv260622554:85 , srv270837987:85 , srv274748606:85 , srv888172191:85 , srv986129162:85 , srv1167640182:86 , srv1237150545:86 , srv1407307746:86 , srv1517347237:86 , srv2032789800:86 , srv426998906:86 , srv433358318:86 , srv690782150:86 , srv834764472:86 , srv842521404:86 , srv998922009:86 , srv1014198383:87 , srv1023728695:87 , srv1293619832:87 , srv1474920935:87 , srv1552315833:87 , srv1582010656:87 , srv1591640422:87 , srv1721379854:87 , srv1854167196:87 , srv1929897788:87 , srv2126988581:87 , srv285398425:87 , srv394870176:87 , srv424471837:87 , srv478018047:87 , srv526831715:87 , srv579455823:87 , srv905051589:87 , srv1829904234:88 , srv1959889283:88 , srv20690662:88 , srv218406726:88 , srv229183957:88 , srv501231148:88 , srv857983825:88 , srv995620768:88 , srv1245739108:89 , srv1398294710:89 , srv1465640366:89 , srv1607997319:89 , srv1843138429:89 , srv1915289404:89 , srv228183504:89 , srv247911853:89 , srv370048868:89 , srv833135100:89 , srv935468523:89 , srv949095833:89 , srv952715834:89 , srv1695875172:90 , srv1998948936:90 , srv30722910:90 , srv397602587:90 , srv635607770:91 , srv148032211:92 , srv1996877870:533 }
2023-03-01 21:54:11,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-01 21:54:11,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148032211=24, srv1090207299=3, srv1465640366=20, srv888172191=89, srv526831715=77, srv1295881204=14, srv1727555733=40, srv986129162=97, srv1249087144=12, srv1510081425=25, srv1023728695=1, srv953111222=95, srv30722910=66, srv1468416506=22, srv328289272=67, srv1293619832=13, srv635607770=80, srv163420653=35, srv1695875172=37, srv690782150=81, srv998922009=99, srv2032789800=53, srv426998906=72, srv1463017994=19, srv1167640182=5, srv1517347237=26, srv1965981817=50, srv143086438=18, srv424471837=71, srv949095833=93, srv1607997319=33, srv247911853=60, srv1014198383=0, srv397602587=70, srv433358318=73, srv955137432=96, srv195273696=48, srv1929897788=47, srv1779411097=42, srv370048868=68, srv1629885558=34, srv834764472=85, srv937424392=92, srv1210838791=7, srv47415013=74, srv995620768=98, srv1843138429=44, srv833135100=84, srv272952275=63, srv218406726=57, srv228183504=58, srv1466106279=21, srv1744644383=41, srv1998948936=52, srv935468523=91, srv1237150545=9, srv1959889283=49, srv905051589=90, srv1591640422=32, srv1854167196=45, srv394870176=69, srv1996877870=51, srv579455823=79, srv1407307746=17, srv1474920935=23, srv229183957=59, srv260622554=61, srv857983825=87, srv113225936=4, srv1679067835=36, srv20690662=54, srv1184754532=6, srv1398294710=16, srv274748606=64, srv1547616082=29, srv1247926960=11, srv1083397957=2, srv842521404=86, srv1915289404=46, srv1552315833=30, srv2126988581=55, srv1534945258=28, srv478018047=75, srv535009071=78, srv1721379854=39, srv2133579536=56, srv805162654=83, srv952715834=94, srv1701983192=38, srv1245739108=10, srv501231148=76, srv270837987=62, srv285398425=65, srv699366386=82, srv1533212309=27, srv887357996=88, srv1353680750=15, srv1582010656=31, srv1232193747=8, srv1829904234=43} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-01 21:54:11,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-01 21:54:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-01 21:54:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-01 21:54:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-01 21:54:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-01 21:54:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-01 21:54:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-01 21:54:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-01 21:54:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-01 21:54:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-01 21:54:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-01 21:54:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-01 21:54:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-01 21:54:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-01 21:54:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-01 21:54:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-01 21:54:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-01 21:54:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-01 21:54:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-01 21:54:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-01 21:54:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-01 21:54:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-01 21:54:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-01 21:54:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-01 21:54:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-01 21:54:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-01 21:54:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-01 21:54:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-01 21:54:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-01 21:54:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-01 21:54:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-01 21:54:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-01 21:54:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-01 21:54:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-01 21:54:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-01 21:54:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-01 21:54:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-01 21:54:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-01 21:54:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-01 21:54:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-01 21:54:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-01 21:54:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-01 21:54:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-01 21:54:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-01 21:54:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-01 21:54:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-01 21:54:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-01 21:54:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-01 21:54:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-01 21:54:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-01 21:54:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-01 21:54:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-01 21:54:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-01 21:54:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-01 21:54:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-01 21:54:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-01 21:54:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-01 21:54:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-01 21:54:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-01 21:54:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-01 21:54:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-01 21:54:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-01 21:54:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-01 21:54:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-01 21:54:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-01 21:54:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-01 21:54:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-01 21:54:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-01 21:54:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-01 21:54:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-01 21:54:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-01 21:54:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-01 21:54:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-01 21:54:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-01 21:54:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-01 21:54:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-01 21:54:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-01 21:54:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-01 21:54:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-01 21:54:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-01 21:54:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-01 21:54:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-01 21:54:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-01 21:54:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-01 21:54:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-01 21:54:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-01 21:54:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-01 21:54:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-01 21:54:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-01 21:54:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-01 21:54:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-01 21:54:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-01 21:54:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-01 21:54:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-01 21:54:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-01 21:54:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-01 21:54:11,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-01 21:54:11,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-01 21:54:11,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-01 21:54:11,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-01 21:54:11,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-01 21:54:11,955 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024531016985917784); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03805601636527022, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024531016985917784); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01375); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.048666666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:11,956 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-01 21:54:11,956 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01704960314859753, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024531016985917784); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03805601636527022, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024531016985917784); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01375); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.048666666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-01 21:54:12,236 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 454 ms to try 5000 different iterations.  Found a solution that moves 1020 regions; Going from a computed imbalance of 0.01704960314859753 to a new imbalance of 8.1160879261172E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0075672979328647244); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008951797592365465); MoveCostFunction : (multiplier=7.0, imbalance=0.11333333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0075672979328647244); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:12,240 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv148032211:81 , srv228183504:82 , srv272952275:82 , srv635607770:82 , srv1295881204:83 , srv1398294710:83 , srv20690662:83 , srv218406726:83 , srv30722910:83 , srv501231148:83 , srv842521404:83 , srv935468523:83 , srv953111222:83 , srv1465640366:84 , srv1582010656:84 , srv1607997319:84 , srv1843138429:84 , srv1915289404:84 , srv1929897788:84 , srv1959889283:84 , srv397602587:84 , srv424471837:84 , srv426998906:84 , srv833135100:84 , srv857983825:84 , srv905051589:84 , srv1014198383:85 , srv1083397957:85 , srv1167640182:85 , srv1237150545:85 , srv1245739108:85 , srv1468416506:85 , srv1591640422:85 , srv1695875172:85 , srv1701983192:85 , srv1829904234:85 , srv195273696:85 , srv1998948936:85 , srv2126988581:85 , srv229183957:85 , srv394870176:85 , srv47415013:85 , srv478018047:85 , srv535009071:85 , srv579455823:85 , srv690782150:85 , srv887357996:85 , srv955137432:85 , srv998922009:85 , srv1023728695:86 , srv1090207299:86 , srv113225936:86 , srv1184754532:86 , srv1293619832:86 , srv1407307746:86 , srv1466106279:86 , srv1533212309:86 , srv1552315833:86 , srv163420653:86 , srv1854167196:86 , srv247911853:86 , srv260622554:86 , srv270837987:86 , srv526831715:86 , srv805162654:86 , srv888172191:86 , srv952715834:86 , srv986129162:86 , srv995620768:86 , srv1247926960:87 , srv1353680750:87 , srv1510081425:87 , srv1517347237:87 , srv1547616082:87 , srv1629885558:87 , srv1679067835:87 , srv1721379854:87 , srv1727555733:87 , srv1965981817:87 , srv2032789800:87 , srv328289272:87 , srv370048868:87 , srv433358318:87 , srv699366386:87 , srv834764472:87 , srv937424392:87 , srv1210838791:88 , srv1249087144:88 , srv143086438:88 , srv1463017994:88 , srv1474920935:88 , srv1534945258:88 , srv1779411097:88 , srv2133579536:88 , srv274748606:88 , srv285398425:88 , srv949095833:88 , srv1232193747:89 , srv1744644383:90 , srv1996877870:533 }
2023-03-01 21:54:12,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-01 21:54:12,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148032211=24, srv1090207299=3, srv1465640366=20, srv888172191=89, srv526831715=77, srv1295881204=14, srv1727555733=40, srv986129162=97, srv1249087144=12, srv1510081425=25, srv1023728695=1, srv953111222=95, srv30722910=66, srv1468416506=22, srv328289272=67, srv1293619832=13, srv635607770=80, srv163420653=35, srv1695875172=37, srv690782150=81, srv998922009=99, srv2032789800=53, srv426998906=72, srv1463017994=19, srv1167640182=5, srv1517347237=26, srv1965981817=50, srv143086438=18, srv424471837=71, srv949095833=93, srv1607997319=33, srv247911853=60, srv1014198383=0, srv397602587=70, srv433358318=73, srv955137432=96, srv195273696=48, srv1929897788=47, srv1779411097=42, srv370048868=68, srv1629885558=34, srv834764472=85, srv937424392=92, srv1210838791=7, srv47415013=74, srv995620768=98, srv1843138429=44, srv833135100=84, srv272952275=63, srv218406726=57, srv228183504=58, srv1466106279=21, srv1744644383=41, srv1998948936=52, srv935468523=91, srv1237150545=9, srv1959889283=49, srv905051589=90, srv1591640422=32, srv1854167196=45, srv394870176=69, srv1996877870=51, srv579455823=79, srv1407307746=17, srv1474920935=23, srv229183957=59, srv260622554=61, srv857983825=87, srv113225936=4, srv1679067835=36, srv20690662=54, srv1184754532=6, srv1398294710=16, srv274748606=64, srv1547616082=29, srv1247926960=11, srv1083397957=2, srv842521404=86, srv1915289404=46, srv1552315833=30, srv2126988581=55, srv1534945258=28, srv478018047=75, srv535009071=78, srv1721379854=39, srv2133579536=56, srv805162654=83, srv952715834=94, srv1701983192=38, srv1245739108=10, srv501231148=76, srv270837987=62, srv285398425=65, srv699366386=82, srv1533212309=27, srv887357996=88, srv1353680750=15, srv1582010656=31, srv1232193747=8, srv1829904234=43} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-01 21:54:12,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-01 21:54:12,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-01 21:54:12,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-01 21:54:12,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-01 21:54:12,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-01 21:54:12,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-01 21:54:12,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-01 21:54:12,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-01 21:54:12,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-01 21:54:12,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-01 21:54:12,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-01 21:54:12,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-01 21:54:12,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-01 21:54:12,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-01 21:54:12,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-01 21:54:12,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-01 21:54:12,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-01 21:54:12,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-01 21:54:12,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-01 21:54:12,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-01 21:54:12,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-01 21:54:12,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-01 21:54:12,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-01 21:54:12,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-01 21:54:12,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-01 21:54:12,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-01 21:54:12,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-01 21:54:12,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-01 21:54:12,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-01 21:54:12,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-01 21:54:12,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-01 21:54:12,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-01 21:54:12,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-01 21:54:12,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-01 21:54:12,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-01 21:54:12,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-01 21:54:12,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-01 21:54:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-01 21:54:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-01 21:54:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-01 21:54:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-01 21:54:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-01 21:54:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-01 21:54:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-01 21:54:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-01 21:54:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-01 21:54:12,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-01 21:54:12,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-01 21:54:12,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-01 21:54:12,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-01 21:54:12,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-01 21:54:12,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-01 21:54:12,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-01 21:54:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-01 21:54:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-01 21:54:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-01 21:54:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-01 21:54:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-01 21:54:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-01 21:54:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-01 21:54:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-01 21:54:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-01 21:54:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-01 21:54:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-01 21:54:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-01 21:54:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-01 21:54:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-01 21:54:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-01 21:54:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-01 21:54:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-01 21:54:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-01 21:54:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-01 21:54:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-01 21:54:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-01 21:54:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-01 21:54:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-01 21:54:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-01 21:54:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-01 21:54:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-01 21:54:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-01 21:54:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-01 21:54:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-01 21:54:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-01 21:54:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-01 21:54:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-01 21:54:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-01 21:54:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-01 21:54:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-01 21:54:12,387 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0075672979328647244); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008951797592365465); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0075672979328647244); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:12,387 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-01 21:54:12,387 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.044665999585757E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0075672979328647244); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008951797592365465); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0075672979328647244); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-01 21:54:12,734 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 445 ms to try 5000 different iterations.  Found a solution that moves 310 regions; Going from a computed imbalance of 8.044665999585757E-4 to a new imbalance of 5.8561222675194666E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.909066927536782E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.034444444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.909066927536782E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:12,736 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1474920935:82 , srv1023728695:83 , srv1247926960:83 , srv1353680750:83 , srv1463017994:83 , srv1014198383:84 , srv1083397957:84 , srv1090207299:84 , srv113225936:84 , srv1210838791:84 , srv1232193747:84 , srv143086438:84 , srv1465640366:84 , srv1466106279:84 , srv1534945258:84 , srv1547616082:84 , srv1591640422:84 , srv163420653:84 , srv1679067835:84 , srv1727555733:84 , srv1744644383:84 , srv1779411097:84 , srv1854167196:84 , srv1915289404:84 , srv1929897788:84 , srv1965981817:84 , srv2126988581:84 , srv2133579536:84 , srv247911853:84 , srv370048868:84 , srv394870176:84 , srv426998906:84 , srv535009071:84 , srv690782150:84 , srv699366386:84 , srv857983825:84 , srv887357996:84 , srv937424392:84 , srv949095833:84 , srv953111222:84 , srv1167640182:85 , srv1184754532:85 , srv1237150545:85 , srv1245739108:85 , srv1249087144:85 , srv1293619832:85 , srv1295881204:85 , srv1398294710:85 , srv1407307746:85 , srv1468416506:85 , srv1510081425:85 , srv1517347237:85 , srv1533212309:85 , srv1552315833:85 , srv1582010656:85 , srv1607997319:85 , srv1629885558:85 , srv1695875172:85 , srv1721379854:85 , srv1829904234:85 , srv1843138429:85 , srv195273696:85 , srv1959889283:85 , srv1998948936:85 , srv2032789800:85 , srv20690662:85 , srv229183957:85 , srv260622554:85 , srv270837987:85 , srv285398425:85 , srv30722910:85 , srv328289272:85 , srv424471837:85 , srv433358318:85 , srv478018047:85 , srv501231148:85 , srv579455823:85 , srv635607770:85 , srv805162654:85 , srv833135100:85 , srv842521404:85 , srv888172191:85 , srv905051589:85 , srv952715834:85 , srv955137432:85 , srv986129162:85 , srv995620768:85 , srv148032211:86 , srv1701983192:86 , srv218406726:86 , srv228183504:86 , srv272952275:86 , srv274748606:86 , srv47415013:86 , srv834764472:86 , srv935468523:86 , srv998922009:86 , srv397602587:87 , srv526831715:87 , srv1996877870:617 }
2023-03-01 21:54:12,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-01 21:54:12,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148032211=24, srv1090207299=3, srv1465640366=20, srv888172191=89, srv526831715=77, srv1295881204=14, srv1727555733=40, srv986129162=97, srv1249087144=12, srv1510081425=25, srv1023728695=1, srv953111222=95, srv30722910=66, srv1468416506=22, srv328289272=67, srv1293619832=13, srv635607770=80, srv163420653=35, srv1695875172=37, srv690782150=81, srv998922009=99, srv2032789800=53, srv426998906=72, srv1463017994=19, srv1167640182=5, srv1517347237=26, srv1965981817=50, srv143086438=18, srv424471837=71, srv949095833=93, srv1607997319=33, srv247911853=60, srv1014198383=0, srv397602587=70, srv433358318=73, srv955137432=96, srv195273696=48, srv1929897788=47, srv1779411097=42, srv370048868=68, srv1629885558=34, srv834764472=85, srv937424392=92, srv1210838791=7, srv47415013=74, srv995620768=98, srv1843138429=44, srv833135100=84, srv272952275=63, srv218406726=57, srv228183504=58, srv1466106279=21, srv1744644383=41, srv1998948936=52, srv935468523=91, srv1237150545=9, srv1959889283=49, srv905051589=90, srv1591640422=32, srv1854167196=45, srv394870176=69, srv1996877870=51, srv579455823=79, srv1407307746=17, srv1474920935=23, srv229183957=59, srv260622554=61, srv857983825=87, srv113225936=4, srv1679067835=36, srv20690662=54, srv1184754532=6, srv1398294710=16, srv274748606=64, srv1547616082=29, srv1247926960=11, srv1083397957=2, srv842521404=86, srv1915289404=46, srv1552315833=30, srv2126988581=55, srv1534945258=28, srv478018047=75, srv535009071=78, srv1721379854=39, srv2133579536=56, srv805162654=83, srv952715834=94, srv1701983192=38, srv1245739108=10, srv501231148=76, srv270837987=62, srv285398425=65, srv699366386=82, srv1533212309=27, srv887357996=88, srv1353680750=15, srv1582010656=31, srv1232193747=8, srv1829904234=43} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-01 21:54:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-01 21:54:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-01 21:54:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-01 21:54:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-01 21:54:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-01 21:54:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-01 21:54:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-01 21:54:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-01 21:54:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-01 21:54:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-01 21:54:12,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-01 21:54:12,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-01 21:54:12,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-01 21:54:12,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-01 21:54:12,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-01 21:54:12,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-01 21:54:12,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-01 21:54:12,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-01 21:54:12,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-01 21:54:12,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-01 21:54:12,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-01 21:54:12,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-01 21:54:12,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-01 21:54:12,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-01 21:54:12,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-01 21:54:12,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-01 21:54:12,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-01 21:54:12,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-01 21:54:12,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-01 21:54:12,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-01 21:54:12,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-01 21:54:12,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-01 21:54:12,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-01 21:54:12,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-01 21:54:12,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-01 21:54:12,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-01 21:54:12,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-01 21:54:12,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-01 21:54:12,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-01 21:54:12,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-01 21:54:12,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-01 21:54:12,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-01 21:54:12,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-01 21:54:12,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-01 21:54:12,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-01 21:54:12,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-01 21:54:12,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-01 21:54:12,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-01 21:54:12,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-01 21:54:12,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-01 21:54:12,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-01 21:54:12,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-01 21:54:12,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-01 21:54:12,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-01 21:54:12,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-01 21:54:12,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-01 21:54:12,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-01 21:54:12,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-01 21:54:12,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-01 21:54:12,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-01 21:54:12,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-01 21:54:12,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-01 21:54:12,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-01 21:54:12,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-01 21:54:12,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-01 21:54:12,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-01 21:54:12,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-01 21:54:12,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-01 21:54:12,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-01 21:54:12,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-01 21:54:12,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-01 21:54:12,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-01 21:54:12,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-01 21:54:12,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-01 21:54:12,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-01 21:54:12,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-01 21:54:12,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-01 21:54:12,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-01 21:54:12,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-01 21:54:12,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-01 21:54:12,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-01 21:54:12,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-01 21:54:12,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-01 21:54:12,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-01 21:54:12,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-01 21:54:12,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-01 21:54:12,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-01 21:54:12,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-01 21:54:12,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-01 21:54:12,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-01 21:54:12,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-01 21:54:12,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-01 21:54:12,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-01 21:54:12,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-01 21:54:12,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-01 21:54:12,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-01 21:54:12,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-01 21:54:12,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-01 21:54:12,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-01 21:54:12,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-01 21:54:12,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-01 21:54:12,894 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.909066927536782E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.909066927536782E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:12,895 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-01 21:54:12,895 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.639055628061152E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.909066927536782E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.909066927536782E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-01 21:54:13,199 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 414 ms to try 5000 different iterations.  Found a solution that moves 40 regions; Going from a computed imbalance of 5.639055628061152E-5 to a new imbalance of 6.678071658667523E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0044444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:13,200 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1249087144:83 , srv143086438:83 , srv1468416506:83 , srv148032211:83 , srv1517347237:83 , srv1701983192:83 , srv1965981817:83 , srv218406726:83 , srv228183504:83 , srv30722910:83 , srv526831715:83 , srv834764472:83 , srv905051589:83 , srv935468523:83 , srv1014198383:84 , srv1023728695:84 , srv1083397957:84 , srv113225936:84 , srv1167640182:84 , srv1184754532:84 , srv1210838791:84 , srv1232193747:84 , srv1237150545:84 , srv1245739108:84 , srv1293619832:84 , srv1295881204:84 , srv1398294710:84 , srv1407307746:84 , srv1465640366:84 , srv1466106279:84 , srv1474920935:84 , srv1510081425:84 , srv1533212309:84 , srv1534945258:84 , srv1547616082:84 , srv1552315833:84 , srv1582010656:84 , srv1591640422:84 , srv1607997319:84 , srv1629885558:84 , srv1679067835:84 , srv1695875172:84 , srv1721379854:84 , srv1727555733:84 , srv1744644383:84 , srv1779411097:84 , srv1843138429:84 , srv1854167196:84 , srv1915289404:84 , srv195273696:84 , srv1959889283:84 , srv1998948936:84 , srv2032789800:84 , srv20690662:84 , srv2126988581:84 , srv260622554:84 , srv270837987:84 , srv272952275:84 , srv274748606:84 , srv285398425:84 , srv328289272:84 , srv370048868:84 , srv394870176:84 , srv397602587:84 , srv424471837:84 , srv426998906:84 , srv433358318:84 , srv47415013:84 , srv478018047:84 , srv535009071:84 , srv635607770:84 , srv690782150:84 , srv699366386:84 , srv805162654:84 , srv833135100:84 , srv842521404:84 , srv887357996:84 , srv888172191:84 , srv937424392:84 , srv949095833:84 , srv952715834:84 , srv955137432:84 , srv986129162:84 , srv995620768:84 , srv998922009:84 , srv1090207299:85 , srv1247926960:85 , srv1353680750:85 , srv1463017994:85 , srv163420653:85 , srv1829904234:85 , srv1929897788:85 , srv2133579536:85 , srv229183957:85 , srv247911853:85 , srv501231148:85 , srv579455823:85 , srv857983825:85 , srv953111222:85 , srv1996877870:684 }
2023-03-01 21:54:13,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-01 21:54:13,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv148032211=24, srv1090207299=3, srv1465640366=20, srv888172191=89, srv526831715=77, srv1295881204=14, srv1727555733=40, srv986129162=97, srv1249087144=12, srv1510081425=25, srv1023728695=1, srv953111222=95, srv30722910=66, srv1468416506=22, srv328289272=67, srv1293619832=13, srv635607770=80, srv163420653=35, srv1695875172=37, srv690782150=81, srv998922009=99, srv2032789800=53, srv426998906=72, srv1463017994=19, srv1167640182=5, srv1517347237=26, srv1965981817=50, srv143086438=18, srv424471837=71, srv949095833=93, srv1607997319=33, srv247911853=60, srv1014198383=0, srv397602587=70, srv433358318=73, srv955137432=96, srv195273696=48, srv1929897788=47, srv1779411097=42, srv370048868=68, srv1629885558=34, srv834764472=85, srv937424392=92, srv1210838791=7, srv47415013=74, srv995620768=98, srv1843138429=44, srv833135100=84, srv272952275=63, srv218406726=57, srv228183504=58, srv1466106279=21, srv1744644383=41, srv1998948936=52, srv935468523=91, srv1237150545=9, srv1959889283=49, srv905051589=90, srv1591640422=32, srv1854167196=45, srv394870176=69, srv1996877870=51, srv579455823=79, srv1407307746=17, srv1474920935=23, srv229183957=59, srv260622554=61, srv857983825=87, srv113225936=4, srv1679067835=36, srv20690662=54, srv1184754532=6, srv1398294710=16, srv274748606=64, srv1547616082=29, srv1247926960=11, srv1083397957=2, srv842521404=86, srv1915289404=46, srv1552315833=30, srv2126988581=55, srv1534945258=28, srv478018047=75, srv535009071=78, srv1721379854=39, srv2133579536=56, srv805162654=83, srv952715834=94, srv1701983192=38, srv1245739108=10, srv501231148=76, srv270837987=62, srv285398425=65, srv699366386=82, srv1533212309=27, srv887357996=88, srv1353680750=15, srv1582010656=31, srv1232193747=8, srv1829904234=43} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-01 21:54:13,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-01 21:54:13,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-01 21:54:13,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-01 21:54:13,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-01 21:54:13,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-01 21:54:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-01 21:54:13,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-01 21:54:13,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-01 21:54:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-01 21:54:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-01 21:54:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-01 21:54:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-01 21:54:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-01 21:54:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-01 21:54:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-01 21:54:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-01 21:54:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-01 21:54:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-01 21:54:13,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-01 21:54:13,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-01 21:54:13,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-01 21:54:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-01 21:54:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-01 21:54:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-01 21:54:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-01 21:54:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-01 21:54:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-01 21:54:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-01 21:54:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-01 21:54:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-01 21:54:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-01 21:54:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-01 21:54:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-01 21:54:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-01 21:54:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-01 21:54:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-01 21:54:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-01 21:54:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-01 21:54:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-01 21:54:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-01 21:54:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-01 21:54:13,360 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:13,361 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1249087144:83 , srv143086438:83 , srv1468416506:83 , srv148032211:83 , srv1517347237:83 , srv1701983192:83 , srv1965981817:83 , srv218406726:83 , srv228183504:83 , srv30722910:83 , srv526831715:83 , srv834764472:83 , srv905051589:83 , srv935468523:83 , srv1014198383:84 , srv1023728695:84 , srv1083397957:84 , srv113225936:84 , srv1167640182:84 , srv1184754532:84 , srv1210838791:84 , srv1232193747:84 , srv1237150545:84 , srv1245739108:84 , srv1293619832:84 , srv1295881204:84 , srv1398294710:84 , srv1407307746:84 , srv1465640366:84 , srv1466106279:84 , srv1474920935:84 , srv1510081425:84 , srv1533212309:84 , srv1534945258:84 , srv1547616082:84 , srv1552315833:84 , srv1582010656:84 , srv1591640422:84 , srv1607997319:84 , srv1629885558:84 , srv1679067835:84 , srv1695875172:84 , srv1721379854:84 , srv1727555733:84 , srv1744644383:84 , srv1779411097:84 , srv1843138429:84 , srv1854167196:84 , srv1915289404:84 , srv195273696:84 , srv1959889283:84 , srv1998948936:84 , srv2032789800:84 , srv20690662:84 , srv2126988581:84 , srv260622554:84 , srv270837987:84 , srv272952275:84 , srv274748606:84 , srv285398425:84 , srv328289272:84 , srv370048868:84 , srv394870176:84 , srv397602587:84 , srv424471837:84 , srv426998906:84 , srv433358318:84 , srv47415013:84 , srv478018047:84 , srv535009071:84 , srv635607770:84 , srv690782150:84 , srv699366386:84 , srv805162654:84 , srv833135100:84 , srv842521404:84 , srv887357996:84 , srv888172191:84 , srv937424392:84 , srv949095833:84 , srv952715834:84 , srv955137432:84 , srv986129162:84 , srv995620768:84 , srv998922009:84 , srv1090207299:85 , srv1247926960:85 , srv1353680750:85 , srv1463017994:85 , srv163420653:85 , srv1829904234:85 , srv1929897788:85 , srv2133579536:85 , srv229183957:85 , srv247911853:85 , srv501231148:85 , srv579455823:85 , srv857983825:85 , srv953111222:85 , srv1996877870:684 }
2023-03-01 21:54:13,391 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=379 (was 360) - SystemLoadAverage LEAK? -, ProcessCount=186 (was 183) - ProcessCount LEAK? -, AvailableMemoryMB=7794 (was 8318)
2023-03-01 21:54:13,401 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=379, ProcessCount=186, AvailableMemoryMB=7794
2023-03-01 21:54:13,402 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-01 21:54:13,402 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-01 21:54:13,413 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1060148249:84 , srv1062887192:84 , srv1250555731:84 , srv135819348:84 , srv1455355134:84 , srv1469119924:84 , srv1527498313:84 , srv1594973124:84 , srv1604772602:84 , srv1607788338:84 , srv1643747905:84 , srv1763592569:84 , srv1777288963:84 , srv1993500657:84 , srv2007274955:84 , srv2060343167:84 , srv2106341479:84 , srv252599780:84 , srv262692305:84 , srv303301249:84 , srv358577743:84 , srv533245965:84 , srv611814128:84 , srv706611288:84 , srv776675891:84 , srv803074978:84 , srv852901949:84 , srv912356884:84 , srv9255887:84 , srv1106516933:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-01 21:54:13,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-01 21:54:13,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1527498313=7, srv803074978=26, srv533245965=22, srv611814128=23, srv2007274955=15, srv852901949=27, srv1594973124=8, srv776675891=25, srv1106516933=2, srv1604772602=9, srv1643747905=11, srv252599780=18, srv262692305=19, srv2060343167=16, srv1469119924=6, srv303301249=20, srv9255887=29, srv1250555731=3, srv1993500657=14, srv135819348=4, srv1763592569=12, srv706611288=24, srv358577743=21, srv1455355134=5, srv2106341479=17, srv1060148249=0, srv912356884=28, srv1777288963=13, srv1607788338=10, srv1062887192=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-01 21:54:13,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-01 21:54:13,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-01 21:54:13,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-01 21:54:13,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-01 21:54:13,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-01 21:54:13,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-01 21:54:13,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-01 21:54:13,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-01 21:54:13,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-01 21:54:13,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-01 21:54:13,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-01 21:54:13,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-01 21:54:13,438 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1021615879015012, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:54:13,438 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912514951091875, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1021615879015012, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-01 21:56:13,426 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6289742 different iterations.  Found a solution that moves 1847 regions; Going from a computed imbalance of 0.9912514951091875 to a new imbalance of 4.3109901406398434E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6840740740740741, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:56:13,434 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1060148249:90 , srv1062887192:90 , srv1106516933:90 , srv1250555731:90 , srv135819348:90 , srv1455355134:90 , srv1469119924:90 , srv1527498313:90 , srv1594973124:90 , srv1604772602:90 , srv1607788338:90 , srv1643747905:90 , srv1763592569:90 , srv1777288963:90 , srv1993500657:90 , srv2007274955:90 , srv2060343167:90 , srv2106341479:90 , srv252599780:90 , srv262692305:90 , srv303301249:90 , srv358577743:90 , srv533245965:90 , srv611814128:90 , srv706611288:90 , srv776675891:90 , srv803074978:90 , srv852901949:90 , srv912356884:90 , srv9255887:90 }
2023-03-01 21:56:13,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-01 21:56:13,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1527498313=7, srv803074978=26, srv533245965=22, srv611814128=23, srv2007274955=15, srv852901949=27, srv1594973124=8, srv776675891=25, srv1106516933=2, srv1604772602=9, srv1643747905=11, srv252599780=18, srv262692305=19, srv2060343167=16, srv1469119924=6, srv303301249=20, srv9255887=29, srv1250555731=3, srv1993500657=14, srv135819348=4, srv1763592569=12, srv706611288=24, srv358577743=21, srv1455355134=5, srv2106341479=17, srv1060148249=0, srv912356884=28, srv1777288963=13, srv1607788338=10, srv1062887192=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-01 21:56:13,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-01 21:56:13,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-01 21:56:13,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-01 21:56:13,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-01 21:56:13,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-01 21:56:13,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-01 21:56:13,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-01 21:56:13,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-01 21:56:13,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-01 21:56:13,451 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-01 21:56:13,452 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1060148249:90 , srv1062887192:90 , srv1106516933:90 , srv1250555731:90 , srv135819348:90 , srv1455355134:90 , srv1469119924:90 , srv1527498313:90 , srv1594973124:90 , srv1604772602:90 , srv1607788338:90 , srv1643747905:90 , srv1763592569:90 , srv1777288963:90 , srv1993500657:90 , srv2007274955:90 , srv2060343167:90 , srv2106341479:90 , srv252599780:90 , srv262692305:90 , srv303301249:90 , srv358577743:90 , srv533245965:90 , srv611814128:90 , srv706611288:90 , srv776675891:90 , srv803074978:90 , srv852901949:90 , srv912356884:90 , srv9255887:90 }
2023-03-01 21:56:13,465 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=188 (was 379), ProcessCount=170 (was 186), AvailableMemoryMB=7802 (was 7794) - AvailableMemoryMB LEAK? -