2023-08-07 16:53:10,199 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-07 16:53:10,521 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-07 16:53:10,638 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-07 16:53:10,705 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-07 16:53:10,715 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-07 16:53:10,730 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=268, ProcessCount=172, AvailableMemoryMB=3335
2023-08-07 16:53:10,732 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-07 16:53:10,732 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-07 16:53:10,876 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1012624073:84 , srv1016305736:84 , srv1024172531:84 , srv1027701367:84 , srv1045122366:84 , srv1059821464:84 , srv1086303135:84 , srv1107064900:84 , srv1122763327:84 , srv1250185438:84 , srv1264737911:84 , srv1362077921:84 , srv1365016038:84 , srv1369655439:84 , srv1387431856:84 , srv1394829616:84 , srv142018859:84 , srv1420201554:84 , srv1425004406:84 , srv1431630771:84 , srv1436164374:84 , srv1442921112:84 , srv1455190117:84 , srv1528267843:84 , srv1537830192:84 , srv1541618195:84 , srv1551887513:84 , srv156981484:84 , srv1586881995:84 , srv1590358390:84 , srv1610064035:84 , srv1611816977:84 , srv1617708292:84 , srv1634455313:84 , srv1649929144:84 , srv1789444823:84 , srv1848430662:84 , srv1879605436:84 , srv1908539288:84 , srv1917725405:84 , srv1939819933:84 , srv1951833300:84 , srv1952846066:84 , srv1964965837:84 , srv1967927774:84 , srv1969166036:84 , srv1969283087:84 , srv2009398026:84 , srv2023301955:84 , srv205931971:84 , srv2109999949:84 , srv2145923511:84 , srv231907047:84 , srv251762143:84 , srv276570358:84 , srv285114589:84 , srv2979383:84 , srv316174979:84 , srv316565515:84 , srv349380321:84 , srv362843471:84 , srv372349414:84 , srv375131146:84 , srv439356508:84 , srv445076658:84 , srv452883514:84 , srv462688533:84 , srv48975180:84 , srv490042548:84 , srv533540182:84 , srv567024201:84 , srv574323373:84 , srv576601501:84 , srv579142208:84 , srv651676461:84 , srv666843927:84 , srv672598221:84 , srv697590176:84 , srv70222881:84 , srv776178812:84 , srv780828314:84 , srv825254541:84 , srv83485115:84 , srv842152:84 , srv848910927:84 , srv874534392:84 , srv879633368:84 , srv881645014:84 , srv889964133:84 , srv896288556:84 , srv897937646:84 , srv918349812:84 , srv921362864:84 , srv921871878:84 , srv92440157:84 , srv947357904:84 , srv968416246:84 , srv972335168:84 , srv976645056:84 , srv1330349599:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-07 16:53:10,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-07 16:53:10,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1610064035=31, srv1634455313=34, srv1951833300=42, srv1848430662=37, srv1365016038=13, srv1330349599=11, srv2979383=57, srv651676461=75, srv1590358390=30, srv1442921112=22, srv1939819933=41, srv349380321=60, srv445076658=65, srv276570358=55, srv1264737911=10, srv1086303135=6, srv316565515=59, srv2009398026=48, srv697590176=78, srv976645056=99, srv1586881995=29, srv1969166036=46, srv142018859=17, srv567024201=71, srv1425004406=19, srv1908539288=39, srv316174979=58, srv874534392=86, srv897937646=91, srv452883514=66, srv1879605436=38, srv1394829616=16, srv2109999949=51, srv848910927=85, srv48975180=68, srv921362864=93, srv1107064900=7, srv918349812=92, srv1045122366=4, srv231907047=53, srv372349414=62, srv251762143=54, srv842152=84, srv490042548=69, srv1387431856=15, srv1541618195=26, srv1967927774=45, srv1537830192=25, srv972335168=98, srv780828314=81, srv1551887513=27, srv1455190117=23, srv968416246=97, srv666843927=76, srv2145923511=52, srv462688533=67, srv285114589=56, srv896288556=90, srv375131146=63, srv1420201554=18, srv205931971=50, srv1059821464=5, srv921871878=94, srv1012624073=0, srv1369655439=14, srv533540182=70, srv1969283087=47, srv1649929144=35, srv1016305736=1, srv1362077921=12, srv2023301955=49, srv579142208=74, srv1917725405=40, srv1789444823=36, srv70222881=79, srv1611816977=32, srv879633368=87, srv1964965837=44, srv1436164374=21, srv889964133=89, srv156981484=28, srv362843471=61, srv825254541=82, srv947357904=96, srv1431630771=20, srv1027701367=3, srv1528267843=24, srv1024172531=2, srv576601501=73, srv672598221=77, srv776178812=80, srv1617708292=33, srv83485115=83, srv881645014=88, srv92440157=95, srv574323373=72, srv1952846066=43, srv1122763327=8, srv439356508=64, srv1250185438=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-07 16:53:10,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-07 16:53:10,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-07 16:53:10,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-07 16:53:10,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-07 16:53:10,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-07 16:53:10,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-07 16:53:10,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-07 16:53:10,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-07 16:53:10,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-07 16:53:10,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-07 16:53:10,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-07 16:53:10,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-07 16:53:10,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-07 16:53:10,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-07 16:53:10,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-07 16:53:10,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-07 16:53:10,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-07 16:53:10,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-07 16:53:10,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-07 16:53:10,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-07 16:53:10,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-07 16:53:10,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-07 16:53:10,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-07 16:53:10,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-07 16:53:10,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-07 16:53:10,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-07 16:53:10,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-07 16:53:10,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-07 16:53:10,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-07 16:53:10,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-07 16:53:10,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-07 16:53:10,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-07 16:53:10,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-07 16:53:10,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-07 16:53:10,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-07 16:53:10,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-07 16:53:10,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-07 16:53:10,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-07 16:53:10,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-07 16:53:10,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-07 16:53:10,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-07 16:53:10,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-07 16:53:10,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-07 16:53:10,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-07 16:53:10,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-07 16:53:10,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-07 16:53:10,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-07 16:53:10,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-07 16:53:10,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-07 16:53:10,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-07 16:53:10,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-07 16:53:10,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-07 16:53:10,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-07 16:53:10,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-07 16:53:10,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-07 16:53:10,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-07 16:53:10,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-07 16:53:10,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-07 16:53:10,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-07 16:53:10,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-07 16:53:10,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-07 16:53:10,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-07 16:53:10,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-07 16:53:10,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-07 16:53:10,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-07 16:53:10,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-07 16:53:10,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-07 16:53:10,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-07 16:53:10,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-07 16:53:10,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-07 16:53:10,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-07 16:53:10,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-07 16:53:10,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-07 16:53:10,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-07 16:53:10,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-07 16:53:10,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-07 16:53:10,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-07 16:53:10,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-07 16:53:10,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-07 16:53:10,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-07 16:53:10,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-07 16:53:10,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-07 16:53:10,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-07 16:53:10,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-07 16:53:10,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-07 16:53:10,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-07 16:53:10,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-07 16:53:10,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-07 16:53:10,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-07 16:53:10,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-07 16:53:10,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-07 16:53:10,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-07 16:53:10,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-07 16:53:10,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-07 16:53:10,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-07 16:53:10,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-07 16:53:10,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-07 16:53:10,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-07 16:53:10,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-07 16:53:10,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-07 16:53:10,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-07 16:53:11,212 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:11,212 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-07 16:53:11,213 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-07 16:53:12,207 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1190 ms to try 5000 different iterations.  Found a solution that moves 4664 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18134797441678815. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03999937647599818, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05379013131338833, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5182222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03999937647599818, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:12,247 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1617708292:81 , srv1024172531:82 , srv1250185438:82 , srv1420201554:82 , srv1436164374:82 , srv490042548:82 , srv574323373:82 , srv881645014:82 , srv896288556:82 , srv1122763327:83 , srv1551887513:83 , srv1611816977:83 , srv1649929144:83 , srv1848430662:83 , srv1917725405:83 , srv205931971:83 , srv897937646:83 , srv921871878:83 , srv1369655439:84 , srv1590358390:84 , srv375131146:84 , srv445076658:84 , srv452883514:84 , srv48975180:84 , srv533540182:84 , srv576601501:84 , srv666843927:84 , srv83485115:84 , srv848910927:84 , srv972335168:84 , srv1027701367:85 , srv1107064900:85 , srv1967927774:85 , srv2009398026:85 , srv2145923511:85 , srv316565515:85 , srv842152:85 , srv947357904:85 , srv1045122366:86 , srv1086303135:86 , srv142018859:86 , srv1431630771:86 , srv1442921112:86 , srv1610064035:86 , srv285114589:86 , srv316174979:86 , srv362843471:86 , srv579142208:86 , srv776178812:86 , srv968416246:86 , srv976645056:86 , srv1016305736:87 , srv1059821464:87 , srv1455190117:87 , srv1528267843:87 , srv2023301955:87 , srv2109999949:87 , srv231907047:87 , srv251762143:87 , srv372349414:87 , srv697590176:87 , srv70222881:87 , srv889964133:87 , srv1012624073:88 , srv1387431856:88 , srv1394829616:88 , srv1425004406:88 , srv156981484:88 , srv1586881995:88 , srv1634455313:88 , srv1789444823:88 , srv1879605436:88 , srv1939819933:88 , srv1951833300:88 , srv349380321:88 , srv439356508:88 , srv462688533:88 , srv567024201:88 , srv651676461:88 , srv825254541:88 , srv879633368:88 , srv918349812:88 , srv1264737911:89 , srv1362077921:89 , srv1365016038:89 , srv1537830192:89 , srv1952846066:89 , srv1969283087:89 , srv92440157:89 , srv1908539288:90 , srv1969166036:90 , srv874534392:90 , srv1541618195:91 , srv276570358:91 , srv672598221:91 , srv780828314:92 , srv1964965837:93 , srv921362864:95 , srv2979383:103 , srv1330349599:445 }
2023-08-07 16:53:12,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-07 16:53:12,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1610064035=31, srv1634455313=34, srv1951833300=42, srv1848430662=37, srv1365016038=13, srv1330349599=11, srv2979383=57, srv651676461=75, srv1590358390=30, srv1442921112=22, srv1939819933=41, srv349380321=60, srv445076658=65, srv276570358=55, srv1264737911=10, srv1086303135=6, srv316565515=59, srv2009398026=48, srv697590176=78, srv976645056=99, srv1586881995=29, srv1969166036=46, srv142018859=17, srv567024201=71, srv1425004406=19, srv1908539288=39, srv316174979=58, srv874534392=86, srv897937646=91, srv452883514=66, srv1879605436=38, srv1394829616=16, srv2109999949=51, srv848910927=85, srv48975180=68, srv921362864=93, srv1107064900=7, srv918349812=92, srv1045122366=4, srv231907047=53, srv372349414=62, srv251762143=54, srv842152=84, srv490042548=69, srv1387431856=15, srv1541618195=26, srv1967927774=45, srv1537830192=25, srv972335168=98, srv780828314=81, srv1551887513=27, srv1455190117=23, srv968416246=97, srv666843927=76, srv2145923511=52, srv462688533=67, srv285114589=56, srv896288556=90, srv375131146=63, srv1420201554=18, srv205931971=50, srv1059821464=5, srv921871878=94, srv1012624073=0, srv1369655439=14, srv533540182=70, srv1969283087=47, srv1649929144=35, srv1016305736=1, srv1362077921=12, srv2023301955=49, srv579142208=74, srv1917725405=40, srv1789444823=36, srv70222881=79, srv1611816977=32, srv879633368=87, srv1964965837=44, srv1436164374=21, srv889964133=89, srv156981484=28, srv362843471=61, srv825254541=82, srv947357904=96, srv1431630771=20, srv1027701367=3, srv1528267843=24, srv1024172531=2, srv576601501=73, srv672598221=77, srv776178812=80, srv1617708292=33, srv83485115=83, srv881645014=88, srv92440157=95, srv574323373=72, srv1952846066=43, srv1122763327=8, srv439356508=64, srv1250185438=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-07 16:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-07 16:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-07 16:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-07 16:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-07 16:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-07 16:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-07 16:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-07 16:53:12,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-07 16:53:12,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-07 16:53:12,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-07 16:53:12,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-07 16:53:12,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-07 16:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-07 16:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-07 16:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-07 16:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-07 16:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-07 16:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-07 16:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-07 16:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-07 16:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-07 16:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-07 16:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-07 16:53:12,431 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03999937647599818, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05379013131338833, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03999937647599818, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:12,431 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-07 16:53:12,432 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18131531639077417, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03999937647599818, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05379013131338833, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03999937647599818, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-07 16:53:12,736 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 430 ms to try 5000 different iterations.  Found a solution that moves 2705 regions; Going from a computed imbalance of 0.18131531639077417 to a new imbalance of 0.017162829857961893. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021488446566176204); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03059088866208045, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.3005555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021488446566176204); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:12,747 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2979383:70 , srv921362864:77 , srv1541618195:80 , srv1964965837:80 , srv1394829616:81 , srv1537830192:81 , srv1908539288:81 , srv1969166036:81 , srv780828314:81 , srv874534392:81 , srv1362077921:82 , srv1365016038:82 , srv1387431856:82 , srv1952846066:82 , srv918349812:82 , srv1264737911:83 , srv1939819933:83 , srv276570358:83 , srv439356508:83 , srv672598221:83 , srv70222881:83 , srv879633368:83 , srv1016305736:84 , srv1059821464:84 , srv1425004406:84 , srv1586881995:84 , srv1610064035:84 , srv1789444823:84 , srv1951833300:84 , srv1969283087:84 , srv251762143:84 , srv372349414:84 , srv462688533:84 , srv776178812:84 , srv1086303135:85 , srv1107064900:85 , srv142018859:85 , srv1455190117:85 , srv1528267843:85 , srv1634455313:85 , srv2023301955:85 , srv362843471:85 , srv825254541:85 , srv968416246:85 , srv1012624073:86 , srv1369655439:86 , srv1431630771:86 , srv156981484:86 , srv1879605436:86 , srv316565515:86 , srv567024201:86 , srv579142208:86 , srv651676461:86 , srv842152:86 , srv92440157:86 , srv976645056:86 , srv1442921112:87 , srv1590358390:87 , srv2109999949:87 , srv231907047:87 , srv316174979:87 , srv349380321:87 , srv452883514:87 , srv697590176:87 , srv1027701367:88 , srv1045122366:88 , srv1436164374:88 , srv1611816977:88 , srv1917725405:88 , srv1967927774:88 , srv2009398026:88 , srv285114589:88 , srv48975180:88 , srv533540182:88 , srv576601501:88 , srv83485115:88 , srv848910927:88 , srv889964133:88 , srv897937646:88 , srv921871878:88 , srv947357904:88 , srv1122763327:89 , srv1420201554:89 , srv1649929144:89 , srv1848430662:89 , srv205931971:89 , srv2145923511:89 , srv375131146:89 , srv490042548:89 , srv666843927:89 , srv881645014:89 , srv972335168:89 , srv1024172531:90 , srv1551887513:90 , srv445076658:90 , srv574323373:90 , srv1250185438:91 , srv896288556:91 , srv1617708292:93 , srv1330349599:520 }
2023-08-07 16:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-07 16:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1610064035=31, srv1634455313=34, srv1951833300=42, srv1848430662=37, srv1365016038=13, srv1330349599=11, srv2979383=57, srv651676461=75, srv1590358390=30, srv1442921112=22, srv1939819933=41, srv349380321=60, srv445076658=65, srv276570358=55, srv1264737911=10, srv1086303135=6, srv316565515=59, srv2009398026=48, srv697590176=78, srv976645056=99, srv1586881995=29, srv1969166036=46, srv142018859=17, srv567024201=71, srv1425004406=19, srv1908539288=39, srv316174979=58, srv874534392=86, srv897937646=91, srv452883514=66, srv1879605436=38, srv1394829616=16, srv2109999949=51, srv848910927=85, srv48975180=68, srv921362864=93, srv1107064900=7, srv918349812=92, srv1045122366=4, srv231907047=53, srv372349414=62, srv251762143=54, srv842152=84, srv490042548=69, srv1387431856=15, srv1541618195=26, srv1967927774=45, srv1537830192=25, srv972335168=98, srv780828314=81, srv1551887513=27, srv1455190117=23, srv968416246=97, srv666843927=76, srv2145923511=52, srv462688533=67, srv285114589=56, srv896288556=90, srv375131146=63, srv1420201554=18, srv205931971=50, srv1059821464=5, srv921871878=94, srv1012624073=0, srv1369655439=14, srv533540182=70, srv1969283087=47, srv1649929144=35, srv1016305736=1, srv1362077921=12, srv2023301955=49, srv579142208=74, srv1917725405=40, srv1789444823=36, srv70222881=79, srv1611816977=32, srv879633368=87, srv1964965837=44, srv1436164374=21, srv889964133=89, srv156981484=28, srv362843471=61, srv825254541=82, srv947357904=96, srv1431630771=20, srv1027701367=3, srv1528267843=24, srv1024172531=2, srv576601501=73, srv672598221=77, srv776178812=80, srv1617708292=33, srv83485115=83, srv881645014=88, srv92440157=95, srv574323373=72, srv1952846066=43, srv1122763327=8, srv439356508=64, srv1250185438=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-07 16:53:12,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-07 16:53:12,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-07 16:53:12,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-07 16:53:12,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-07 16:53:12,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-07 16:53:12,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-07 16:53:12,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-07 16:53:12,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-07 16:53:12,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-07 16:53:12,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-07 16:53:12,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-07 16:53:12,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-07 16:53:12,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-07 16:53:12,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-07 16:53:12,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-07 16:53:12,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-07 16:53:12,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-07 16:53:12,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-07 16:53:12,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-07 16:53:12,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-07 16:53:12,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-07 16:53:12,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-07 16:53:12,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-07 16:53:12,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-07 16:53:12,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-07 16:53:12,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-07 16:53:12,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-07 16:53:12,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-07 16:53:12,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-07 16:53:12,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-07 16:53:12,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-07 16:53:12,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-07 16:53:12,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-07 16:53:12,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-07 16:53:12,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-07 16:53:12,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-07 16:53:12,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-07 16:53:12,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-07 16:53:12,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-07 16:53:12,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-07 16:53:12,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-07 16:53:12,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-07 16:53:12,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-07 16:53:12,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-07 16:53:12,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-07 16:53:12,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-07 16:53:12,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-07 16:53:12,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-07 16:53:12,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-07 16:53:12,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-07 16:53:12,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-07 16:53:12,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-07 16:53:12,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-07 16:53:12,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-07 16:53:12,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-07 16:53:12,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-07 16:53:12,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-07 16:53:12,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-07 16:53:12,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-07 16:53:12,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-07 16:53:12,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-07 16:53:12,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-07 16:53:12,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-07 16:53:12,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-07 16:53:12,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-07 16:53:12,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-07 16:53:12,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-07 16:53:12,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-07 16:53:12,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-07 16:53:12,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-07 16:53:12,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-07 16:53:12,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-07 16:53:12,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-07 16:53:12,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-07 16:53:12,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-07 16:53:12,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-07 16:53:12,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-07 16:53:12,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-07 16:53:12,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-07 16:53:12,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-07 16:53:12,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-07 16:53:12,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-07 16:53:12,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-07 16:53:12,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-07 16:53:12,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-07 16:53:12,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-07 16:53:12,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-07 16:53:12,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-07 16:53:12,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-07 16:53:12,932 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021488446566176204); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03059088866208045, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021488446566176204); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:12,932 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-07 16:53:12,933 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01714388904313174, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021488446566176204); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03059088866208045, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021488446566176204); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-07 16:53:13,286 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 484 ms to try 5000 different iterations.  Found a solution that moves 1014 regions; Going from a computed imbalance of 0.01714388904313174 to a new imbalance of 0.0010068307290904057. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0054249519356559806); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0079561085525314); MoveCostFunction : (multiplier=7.0, imbalance=0.11266666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0054249519356559806); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:13,288 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv92440157:82 , srv1045122366:83 , srv156981484:83 , srv1879605436:83 , srv276570358:83 , srv285114589:83 , srv349380321:83 , srv445076658:83 , srv567024201:83 , srv672598221:83 , srv697590176:83 , srv889964133:83 , srv1250185438:84 , srv1442921112:84 , srv1964965837:84 , srv1969283087:84 , srv2109999949:84 , srv2145923511:84 , srv231907047:84 , srv375131146:84 , srv651676461:84 , srv825254541:84 , srv896288556:84 , srv947357904:84 , srv1012624073:85 , srv1027701367:85 , srv1122763327:85 , srv1425004406:85 , srv1431630771:85 , srv1551887513:85 , srv1634455313:85 , srv1649929144:85 , srv1848430662:85 , srv1951833300:85 , srv1967927774:85 , srv2009398026:85 , srv2979383:85 , srv316174979:85 , srv462688533:85 , srv579142208:85 , srv666843927:85 , srv780828314:85 , srv83485115:85 , srv972335168:85 , srv976645056:85 , srv1016305736:86 , srv1024172531:86 , srv1264737911:86 , srv1362077921:86 , srv1365016038:86 , srv142018859:86 , srv1420201554:86 , srv1455190117:86 , srv1528267843:86 , srv1541618195:86 , srv1586881995:86 , srv1617708292:86 , srv1789444823:86 , srv1908539288:86 , srv1917725405:86 , srv1952846066:86 , srv2023301955:86 , srv205931971:86 , srv362843471:86 , srv439356508:86 , srv48975180:86 , srv490042548:86 , srv533540182:86 , srv574323373:86 , srv576601501:86 , srv842152:86 , srv848910927:86 , srv881645014:86 , srv897937646:86 , srv921362864:86 , srv968416246:86 , srv1059821464:87 , srv1086303135:87 , srv1107064900:87 , srv1387431856:87 , srv1537830192:87 , srv1590358390:87 , srv1610064035:87 , srv1611816977:87 , srv1939819933:87 , srv1969166036:87 , srv251762143:87 , srv316565515:87 , srv372349414:87 , srv452883514:87 , srv70222881:87 , srv776178812:87 , srv874534392:87 , srv879633368:87 , srv918349812:87 , srv921871878:87 , srv1369655439:88 , srv1436164374:88 , srv1394829616:89 , srv1330349599:541 }
2023-08-07 16:53:13,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-07 16:53:13,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1610064035=31, srv1634455313=34, srv1951833300=42, srv1848430662=37, srv1365016038=13, srv1330349599=11, srv2979383=57, srv651676461=75, srv1590358390=30, srv1442921112=22, srv1939819933=41, srv349380321=60, srv445076658=65, srv276570358=55, srv1264737911=10, srv1086303135=6, srv316565515=59, srv2009398026=48, srv697590176=78, srv976645056=99, srv1586881995=29, srv1969166036=46, srv142018859=17, srv567024201=71, srv1425004406=19, srv1908539288=39, srv316174979=58, srv874534392=86, srv897937646=91, srv452883514=66, srv1879605436=38, srv1394829616=16, srv2109999949=51, srv848910927=85, srv48975180=68, srv921362864=93, srv1107064900=7, srv918349812=92, srv1045122366=4, srv231907047=53, srv372349414=62, srv251762143=54, srv842152=84, srv490042548=69, srv1387431856=15, srv1541618195=26, srv1967927774=45, srv1537830192=25, srv972335168=98, srv780828314=81, srv1551887513=27, srv1455190117=23, srv968416246=97, srv666843927=76, srv2145923511=52, srv462688533=67, srv285114589=56, srv896288556=90, srv375131146=63, srv1420201554=18, srv205931971=50, srv1059821464=5, srv921871878=94, srv1012624073=0, srv1369655439=14, srv533540182=70, srv1969283087=47, srv1649929144=35, srv1016305736=1, srv1362077921=12, srv2023301955=49, srv579142208=74, srv1917725405=40, srv1789444823=36, srv70222881=79, srv1611816977=32, srv879633368=87, srv1964965837=44, srv1436164374=21, srv889964133=89, srv156981484=28, srv362843471=61, srv825254541=82, srv947357904=96, srv1431630771=20, srv1027701367=3, srv1528267843=24, srv1024172531=2, srv576601501=73, srv672598221=77, srv776178812=80, srv1617708292=33, srv83485115=83, srv881645014=88, srv92440157=95, srv574323373=72, srv1952846066=43, srv1122763327=8, srv439356508=64, srv1250185438=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-07 16:53:13,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-07 16:53:13,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-07 16:53:13,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-07 16:53:13,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-07 16:53:13,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-07 16:53:13,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-07 16:53:13,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-07 16:53:13,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-07 16:53:13,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-07 16:53:13,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-07 16:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-07 16:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-07 16:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-07 16:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-07 16:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-07 16:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-07 16:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-07 16:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-07 16:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-07 16:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-07 16:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-07 16:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-07 16:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-07 16:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-07 16:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-07 16:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-07 16:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-07 16:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-07 16:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-07 16:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-07 16:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-07 16:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-07 16:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-07 16:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-07 16:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-07 16:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-07 16:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-07 16:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-07 16:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-07 16:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-07 16:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-07 16:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-07 16:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-07 16:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-07 16:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-07 16:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-07 16:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-07 16:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-07 16:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-07 16:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-07 16:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-07 16:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-07 16:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-07 16:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-07 16:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-07 16:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-07 16:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-07 16:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-07 16:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-07 16:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-07 16:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-07 16:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-07 16:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-07 16:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-07 16:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-07 16:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-07 16:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-07 16:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-07 16:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-07 16:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-07 16:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-07 16:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-07 16:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-07 16:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-07 16:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-07 16:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-07 16:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-07 16:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-07 16:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-07 16:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-07 16:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-07 16:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-07 16:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-07 16:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-07 16:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-07 16:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-07 16:53:13,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-07 16:53:13,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-07 16:53:13,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-07 16:53:13,454 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0054249519356559806); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0079561085525314); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0054249519356559806); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:13,454 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-07 16:53:13,455 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=9.997305493352209E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0054249519356559806); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0079561085525314); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0054249519356559806); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-07 16:53:13,785 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 447 ms to try 5000 different iterations.  Found a solution that moves 255 regions; Going from a computed imbalance of 9.997305493352209E-4 to a new imbalance of 2.747224355765133E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.028333333333333332, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:13,787 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1617708292:82 , srv1012624073:83 , srv1059821464:83 , srv205931971:83 , srv848910927:83 , srv1024172531:84 , srv1027701367:84 , srv1086303135:84 , srv1264737911:84 , srv1369655439:84 , srv1394829616:84 , srv1436164374:84 , srv1455190117:84 , srv1528267843:84 , srv1551887513:84 , srv1586881995:84 , srv1590358390:84 , srv1611816977:84 , srv1789444823:84 , srv1908539288:84 , srv1939819933:84 , srv1969166036:84 , srv1969283087:84 , srv2009398026:84 , srv2023301955:84 , srv2109999949:84 , srv2145923511:84 , srv231907047:84 , srv251762143:84 , srv276570358:84 , srv2979383:84 , srv316565515:84 , srv349380321:84 , srv372349414:84 , srv452883514:84 , srv48975180:84 , srv533540182:84 , srv567024201:84 , srv574323373:84 , srv576601501:84 , srv651676461:84 , srv666843927:84 , srv70222881:84 , srv780828314:84 , srv874534392:84 , srv879633368:84 , srv889964133:84 , srv921362864:84 , srv921871878:84 , srv972335168:84 , srv1016305736:85 , srv1045122366:85 , srv1107064900:85 , srv1122763327:85 , srv1362077921:85 , srv1365016038:85 , srv1387431856:85 , srv142018859:85 , srv1420201554:85 , srv1425004406:85 , srv1431630771:85 , srv1537830192:85 , srv1541618195:85 , srv156981484:85 , srv1610064035:85 , srv1634455313:85 , srv1649929144:85 , srv1848430662:85 , srv1879605436:85 , srv1917725405:85 , srv1951833300:85 , srv1952846066:85 , srv1964965837:85 , srv1967927774:85 , srv285114589:85 , srv316174979:85 , srv362843471:85 , srv375131146:85 , srv439356508:85 , srv445076658:85 , srv462688533:85 , srv490042548:85 , srv579142208:85 , srv672598221:85 , srv697590176:85 , srv825254541:85 , srv83485115:85 , srv842152:85 , srv881645014:85 , srv897937646:85 , srv918349812:85 , srv947357904:85 , srv968416246:85 , srv976645056:85 , srv1250185438:86 , srv1442921112:86 , srv776178812:86 , srv896288556:86 , srv92440157:86 , srv1330349599:636 }
2023-08-07 16:53:13,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-07 16:53:13,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1610064035=31, srv1634455313=34, srv1951833300=42, srv1848430662=37, srv1365016038=13, srv1330349599=11, srv2979383=57, srv651676461=75, srv1590358390=30, srv1442921112=22, srv1939819933=41, srv349380321=60, srv445076658=65, srv276570358=55, srv1264737911=10, srv1086303135=6, srv316565515=59, srv2009398026=48, srv697590176=78, srv976645056=99, srv1586881995=29, srv1969166036=46, srv142018859=17, srv567024201=71, srv1425004406=19, srv1908539288=39, srv316174979=58, srv874534392=86, srv897937646=91, srv452883514=66, srv1879605436=38, srv1394829616=16, srv2109999949=51, srv848910927=85, srv48975180=68, srv921362864=93, srv1107064900=7, srv918349812=92, srv1045122366=4, srv231907047=53, srv372349414=62, srv251762143=54, srv842152=84, srv490042548=69, srv1387431856=15, srv1541618195=26, srv1967927774=45, srv1537830192=25, srv972335168=98, srv780828314=81, srv1551887513=27, srv1455190117=23, srv968416246=97, srv666843927=76, srv2145923511=52, srv462688533=67, srv285114589=56, srv896288556=90, srv375131146=63, srv1420201554=18, srv205931971=50, srv1059821464=5, srv921871878=94, srv1012624073=0, srv1369655439=14, srv533540182=70, srv1969283087=47, srv1649929144=35, srv1016305736=1, srv1362077921=12, srv2023301955=49, srv579142208=74, srv1917725405=40, srv1789444823=36, srv70222881=79, srv1611816977=32, srv879633368=87, srv1964965837=44, srv1436164374=21, srv889964133=89, srv156981484=28, srv362843471=61, srv825254541=82, srv947357904=96, srv1431630771=20, srv1027701367=3, srv1528267843=24, srv1024172531=2, srv576601501=73, srv672598221=77, srv776178812=80, srv1617708292=33, srv83485115=83, srv881645014=88, srv92440157=95, srv574323373=72, srv1952846066=43, srv1122763327=8, srv439356508=64, srv1250185438=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-07 16:53:13,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-07 16:53:13,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-07 16:53:13,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-07 16:53:13,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-07 16:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-07 16:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-07 16:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-07 16:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-07 16:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-07 16:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-07 16:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-07 16:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-07 16:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-07 16:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-07 16:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-07 16:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-07 16:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-07 16:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-07 16:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-07 16:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-07 16:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-07 16:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-07 16:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-07 16:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-07 16:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-07 16:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-07 16:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-07 16:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-07 16:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-07 16:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-07 16:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-07 16:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-07 16:53:13,948 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:13,948 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-07 16:53:13,949 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.56866953943652E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-07 16:53:14,208 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 376 ms to try 5000 different iterations.  Found a solution that moves 24 regions; Going from a computed imbalance of 2.56866953943652E-5 to a new imbalance of 6.566037264108393E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0026666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:14,209 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1250185438:83 , srv1442921112:83 , srv1634455313:83 , srv1967927774:83 , srv316174979:83 , srv776178812:83 , srv896288556:83 , srv92440157:83 , srv1012624073:84 , srv1016305736:84 , srv1024172531:84 , srv1027701367:84 , srv1045122366:84 , srv1086303135:84 , srv1107064900:84 , srv1122763327:84 , srv1264737911:84 , srv1362077921:84 , srv1365016038:84 , srv1369655439:84 , srv1387431856:84 , srv1394829616:84 , srv142018859:84 , srv1420201554:84 , srv1425004406:84 , srv1431630771:84 , srv1436164374:84 , srv1455190117:84 , srv1528267843:84 , srv1537830192:84 , srv1541618195:84 , srv1551887513:84 , srv156981484:84 , srv1586881995:84 , srv1590358390:84 , srv1610064035:84 , srv1611816977:84 , srv1617708292:84 , srv1649929144:84 , srv1789444823:84 , srv1848430662:84 , srv1879605436:84 , srv1917725405:84 , srv1939819933:84 , srv1951833300:84 , srv1952846066:84 , srv1964965837:84 , srv1969166036:84 , srv2009398026:84 , srv2023301955:84 , srv2109999949:84 , srv2145923511:84 , srv231907047:84 , srv251762143:84 , srv285114589:84 , srv2979383:84 , srv316565515:84 , srv349380321:84 , srv362843471:84 , srv372349414:84 , srv375131146:84 , srv439356508:84 , srv445076658:84 , srv452883514:84 , srv462688533:84 , srv48975180:84 , srv490042548:84 , srv533540182:84 , srv574323373:84 , srv576601501:84 , srv579142208:84 , srv651676461:84 , srv666843927:84 , srv672598221:84 , srv697590176:84 , srv780828314:84 , srv825254541:84 , srv83485115:84 , srv842152:84 , srv874534392:84 , srv879633368:84 , srv881645014:84 , srv889964133:84 , srv897937646:84 , srv918349812:84 , srv921362864:84 , srv921871878:84 , srv947357904:84 , srv968416246:84 , srv972335168:84 , srv976645056:84 , srv1059821464:85 , srv1908539288:85 , srv1969283087:85 , srv205931971:85 , srv276570358:85 , srv567024201:85 , srv70222881:85 , srv848910927:85 , srv1330349599:684 }
2023-08-07 16:53:14,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-07 16:53:14,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1610064035=31, srv1634455313=34, srv1951833300=42, srv1848430662=37, srv1365016038=13, srv1330349599=11, srv2979383=57, srv651676461=75, srv1590358390=30, srv1442921112=22, srv1939819933=41, srv349380321=60, srv445076658=65, srv276570358=55, srv1264737911=10, srv1086303135=6, srv316565515=59, srv2009398026=48, srv697590176=78, srv976645056=99, srv1586881995=29, srv1969166036=46, srv142018859=17, srv567024201=71, srv1425004406=19, srv1908539288=39, srv316174979=58, srv874534392=86, srv897937646=91, srv452883514=66, srv1879605436=38, srv1394829616=16, srv2109999949=51, srv848910927=85, srv48975180=68, srv921362864=93, srv1107064900=7, srv918349812=92, srv1045122366=4, srv231907047=53, srv372349414=62, srv251762143=54, srv842152=84, srv490042548=69, srv1387431856=15, srv1541618195=26, srv1967927774=45, srv1537830192=25, srv972335168=98, srv780828314=81, srv1551887513=27, srv1455190117=23, srv968416246=97, srv666843927=76, srv2145923511=52, srv462688533=67, srv285114589=56, srv896288556=90, srv375131146=63, srv1420201554=18, srv205931971=50, srv1059821464=5, srv921871878=94, srv1012624073=0, srv1369655439=14, srv533540182=70, srv1969283087=47, srv1649929144=35, srv1016305736=1, srv1362077921=12, srv2023301955=49, srv579142208=74, srv1917725405=40, srv1789444823=36, srv70222881=79, srv1611816977=32, srv879633368=87, srv1964965837=44, srv1436164374=21, srv889964133=89, srv156981484=28, srv362843471=61, srv825254541=82, srv947357904=96, srv1431630771=20, srv1027701367=3, srv1528267843=24, srv1024172531=2, srv576601501=73, srv672598221=77, srv776178812=80, srv1617708292=33, srv83485115=83, srv881645014=88, srv92440157=95, srv574323373=72, srv1952846066=43, srv1122763327=8, srv439356508=64, srv1250185438=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-07 16:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-07 16:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-07 16:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-07 16:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-07 16:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-07 16:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-07 16:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-07 16:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-07 16:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-07 16:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-07 16:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-07 16:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-07 16:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-07 16:53:14,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-07 16:53:14,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-07 16:53:14,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-07 16:53:14,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-07 16:53:14,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-07 16:53:14,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-07 16:53:14,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-07 16:53:14,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-07 16:53:14,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-07 16:53:14,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-07 16:53:14,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-07 16:53:14,362 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:14,363 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1250185438:83 , srv1442921112:83 , srv1634455313:83 , srv1967927774:83 , srv316174979:83 , srv776178812:83 , srv896288556:83 , srv92440157:83 , srv1012624073:84 , srv1016305736:84 , srv1024172531:84 , srv1027701367:84 , srv1045122366:84 , srv1086303135:84 , srv1107064900:84 , srv1122763327:84 , srv1264737911:84 , srv1362077921:84 , srv1365016038:84 , srv1369655439:84 , srv1387431856:84 , srv1394829616:84 , srv142018859:84 , srv1420201554:84 , srv1425004406:84 , srv1431630771:84 , srv1436164374:84 , srv1455190117:84 , srv1528267843:84 , srv1537830192:84 , srv1541618195:84 , srv1551887513:84 , srv156981484:84 , srv1586881995:84 , srv1590358390:84 , srv1610064035:84 , srv1611816977:84 , srv1617708292:84 , srv1649929144:84 , srv1789444823:84 , srv1848430662:84 , srv1879605436:84 , srv1917725405:84 , srv1939819933:84 , srv1951833300:84 , srv1952846066:84 , srv1964965837:84 , srv1969166036:84 , srv2009398026:84 , srv2023301955:84 , srv2109999949:84 , srv2145923511:84 , srv231907047:84 , srv251762143:84 , srv285114589:84 , srv2979383:84 , srv316565515:84 , srv349380321:84 , srv362843471:84 , srv372349414:84 , srv375131146:84 , srv439356508:84 , srv445076658:84 , srv452883514:84 , srv462688533:84 , srv48975180:84 , srv490042548:84 , srv533540182:84 , srv574323373:84 , srv576601501:84 , srv579142208:84 , srv651676461:84 , srv666843927:84 , srv672598221:84 , srv697590176:84 , srv780828314:84 , srv825254541:84 , srv83485115:84 , srv842152:84 , srv874534392:84 , srv879633368:84 , srv881645014:84 , srv889964133:84 , srv897937646:84 , srv918349812:84 , srv921362864:84 , srv921871878:84 , srv947357904:84 , srv968416246:84 , srv972335168:84 , srv976645056:84 , srv1059821464:85 , srv1908539288:85 , srv1969283087:85 , srv205931971:85 , srv276570358:85 , srv567024201:85 , srv70222881:85 , srv848910927:85 , srv1330349599:684 }
2023-08-07 16:53:14,397 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=311 (was 268) - SystemLoadAverage LEAK? -, ProcessCount=173 (was 172) - ProcessCount LEAK? -, AvailableMemoryMB=2447 (was 3335)
2023-08-07 16:53:14,405 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=311, ProcessCount=173, AvailableMemoryMB=2445
2023-08-07 16:53:14,406 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-07 16:53:14,407 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-07 16:53:14,421 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002594646:84 , srv1132369930:84 , srv1212200919:84 , srv1224852917:84 , srv1262347336:84 , srv1287083066:84 , srv1299954121:84 , srv1339724439:84 , srv1344278909:84 , srv1507755766:84 , srv1571195085:84 , srv1613136659:84 , srv1628676931:84 , srv1678841941:84 , srv1732276422:84 , srv1839932507:84 , srv1856733146:84 , srv1977283276:84 , srv2024419781:84 , srv2090230829:84 , srv236838280:84 , srv276808855:84 , srv295336772:84 , srv372590073:84 , srv491632608:84 , srv770003865:84 , srv880482934:84 , srv929285714:84 , srv979140297:84 , srv748050761:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-07 16:53:14,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-07 16:53:14,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1212200919=2, srv1977283276=17, srv2090230829=19, srv276808855=21, srv880482934=27, srv1507755766=9, srv929285714=28, srv1571195085=10, srv372590073=23, srv1732276422=14, srv1613136659=11, srv1224852917=3, srv1839932507=15, srv491632608=24, srv748050761=25, srv2024419781=18, srv1287083066=5, srv1002594646=0, srv1856733146=16, srv1339724439=7, srv236838280=20, srv295336772=22, srv1299954121=6, srv1132369930=1, srv1344278909=8, srv1262347336=4, srv979140297=29, srv1628676931=12, srv1678841941=13, srv770003865=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-07 16:53:14,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-07 16:53:14,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-07 16:53:14,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-07 16:53:14,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-07 16:53:14,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-07 16:53:14,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-07 16:53:14,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-07 16:53:14,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-07 16:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-07 16:53:14,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-07 16:53:14,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-07 16:53:14,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-07 16:53:14,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-07 16:53:14,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-07 16:53:14,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-07 16:53:14,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-07 16:53:14,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-07 16:53:14,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-07 16:53:14,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-07 16:53:14,468 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0935662305117873, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:53:14,468 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912487867401404, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0935662305117873, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-07 16:55:14,444 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6374968 different iterations.  Found a solution that moves 1864 regions; Going from a computed imbalance of 0.9912487867401404 to a new imbalance of 4.3506689887128686E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6903703703703704, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:55:14,450 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1002594646:90 , srv1132369930:90 , srv1212200919:90 , srv1224852917:90 , srv1262347336:90 , srv1287083066:90 , srv1299954121:90 , srv1339724439:90 , srv1344278909:90 , srv1507755766:90 , srv1571195085:90 , srv1613136659:90 , srv1628676931:90 , srv1678841941:90 , srv1732276422:90 , srv1839932507:90 , srv1856733146:90 , srv1977283276:90 , srv2024419781:90 , srv2090230829:90 , srv236838280:90 , srv276808855:90 , srv295336772:90 , srv372590073:90 , srv491632608:90 , srv748050761:90 , srv770003865:90 , srv880482934:90 , srv929285714:90 , srv979140297:90 }
2023-08-07 16:55:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-07 16:55:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1212200919=2, srv1977283276=17, srv2090230829=19, srv276808855=21, srv880482934=27, srv1507755766=9, srv929285714=28, srv1571195085=10, srv372590073=23, srv1732276422=14, srv1613136659=11, srv1224852917=3, srv1839932507=15, srv491632608=24, srv748050761=25, srv2024419781=18, srv1287083066=5, srv1002594646=0, srv1856733146=16, srv1339724439=7, srv236838280=20, srv295336772=22, srv1299954121=6, srv1132369930=1, srv1344278909=8, srv1262347336=4, srv979140297=29, srv1628676931=12, srv1678841941=13, srv770003865=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-07 16:55:14,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-07 16:55:14,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-07 16:55:14,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-07 16:55:14,473 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-07 16:55:14,473 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1002594646:90 , srv1132369930:90 , srv1212200919:90 , srv1224852917:90 , srv1262347336:90 , srv1287083066:90 , srv1299954121:90 , srv1339724439:90 , srv1344278909:90 , srv1507755766:90 , srv1571195085:90 , srv1613136659:90 , srv1628676931:90 , srv1678841941:90 , srv1732276422:90 , srv1839932507:90 , srv1856733146:90 , srv1977283276:90 , srv2024419781:90 , srv2090230829:90 , srv236838280:90 , srv276808855:90 , srv295336772:90 , srv372590073:90 , srv491632608:90 , srv748050761:90 , srv770003865:90 , srv880482934:90 , srv929285714:90 , srv979140297:90 }
2023-08-07 16:55:14,491 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=225 (was 311), ProcessCount=171 (was 173), AvailableMemoryMB=2622 (was 2445) - AvailableMemoryMB LEAK? -