2024-04-10 17:00:52,809 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-04-10 17:00:53,247 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-04-10 17:00:54,038 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-04-10 17:00:54,195 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-10 17:00:54,245 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-10 17:00:54,266 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=11547, ProcessCount=191, AvailableMemoryMB=14903
2024-04-10 17:00:54,268 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-10 17:00:54,268 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-10 17:00:54,439 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1018880132:84 , srv1032434426:84 , srv1071342175:84 , srv1081147243:84 , srv1098543822:84 , srv1110155236:84 , srv1131600258:84 , srv1141145125:84 , srv1147520937:84 , srv11713249:84 , srv1202360808:84 , srv1223740607:84 , srv1225036245:84 , srv1235067098:84 , srv1235269320:84 , srv1273842748:84 , srv1278746631:84 , srv1296678761:84 , srv1329037891:84 , srv1456970655:84 , srv1472078778:84 , srv1503556082:84 , srv1515232659:84 , srv1522639362:84 , srv1527574346:84 , srv1533009500:84 , srv1539973642:84 , srv1563399521:84 , srv1568773295:84 , srv1589059638:84 , srv1589659058:84 , srv1595606162:84 , srv1619809549:84 , srv1655265206:84 , srv1657757764:84 , srv1671627112:84 , srv1683513782:84 , srv169412800:84 , srv1706005018:84 , srv1724419019:84 , srv1742762484:84 , srv1784549311:84 , srv1791398501:84 , srv1827503831:84 , srv1895918247:84 , srv195127816:84 , srv1956388418:84 , srv1967450335:84 , srv1983114901:84 , srv1985614646:84 , srv1987989601:84 , srv1991445297:84 , srv1993424891:84 , srv2025832398:84 , srv206161647:84 , srv2070195560:84 , srv2109566810:84 , srv2114399213:84 , srv2126001641:84 , srv2146471787:84 , srv231713070:84 , srv262560732:84 , srv270469601:84 , srv273571781:84 , srv284755449:84 , srv291943529:84 , srv315422803:84 , srv316941744:84 , srv358144764:84 , srv370409461:84 , srv392808494:84 , srv428856455:84 , srv43116366:84 , srv465674193:84 , srv514260439:84 , srv526931055:84 , srv555164897:84 , srv582869661:84 , srv584240323:84 , srv618626462:84 , srv629486715:84 , srv629735961:84 , srv634831444:84 , srv646346391:84 , srv675407332:84 , srv712377978:84 , srv719799626:84 , srv760181794:84 , srv770157472:84 , srv770899484:84 , srv780068127:84 , srv791148546:84 , srv794315580:84 , srv831706626:84 , srv845731004:84 , srv860708720:84 , srv906061938:84 , srv913645354:84 , srv987285979:84 , srv1764147544:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-04-10 17:00:54,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 17:00:54,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1791398501=43, srv358144764=69, srv392808494=71, srv584240323=79, srv634831444=83, srv794315580=93, srv2126001641=59, srv1563399521=27, srv526931055=76, srv1472078778=20, srv1273842748=15, srv1018880132=0, srv1533009500=25, srv1683513782=36, srv1657757764=34, srv1655265206=33, srv1296678761=17, srv195127816=46, srv316941744=68, srv1987989601=51, srv1131600258=6, srv712377978=86, srv514260439=75, srv906061938=97, srv675407332=85, srv1568773295=28, srv262560732=62, srv2114399213=58, srv1993424891=53, srv1225036245=12, srv629486715=81, srv987285979=99, srv1956388418=47, srv1071342175=2, srv1527574346=24, srv760181794=88, srv1967450335=48, srv1589059638=29, srv428856455=72, srv1110155236=5, srv1706005018=38, srv1141145125=7, srv169412800=37, srv1032434426=1, srv1827503831=44, srv780068127=91, srv860708720=96, srv629735961=82, srv1098543822=4, srv273571781=64, srv2109566810=57, srv43116366=73, srv1619809549=32, srv1456970655=19, srv284755449=65, srv770899484=90, srv1539973642=26, srv1589659058=30, srv1784549311=42, srv831706626=94, srv1595606162=31, srv555164897=77, srv231713070=61, srv1278746631=16, srv582869661=78, srv270469601=63, srv646346391=84, srv1522639362=23, srv1983114901=49, srv770157472=89, srv1223740607=11, srv1235067098=13, srv2025832398=54, srv1742762484=40, srv1329037891=18, srv291943529=66, srv1515232659=22, srv1671627112=35, srv845731004=95, srv1895918247=45, srv1985614646=50, srv1503556082=21, srv1202360808=10, srv618626462=80, srv2146471787=60, srv11713249=9, srv1081147243=3, srv1764147544=41, srv465674193=74, srv1147520937=8, srv1991445297=52, srv791148546=92, srv719799626=87, srv2070195560=56, srv1235269320=14, srv370409461=70, srv913645354=98, srv1724419019=39, srv206161647=55, srv315422803=67} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 17:00:54,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 17:00:54,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 17:00:54,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 17:00:54,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 17:00:54,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 17:00:54,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 17:00:54,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 17:00:54,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 17:00:54,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 17:00:54,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 17:00:54,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 17:00:54,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 17:00:54,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 17:00:54,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 17:00:54,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 17:00:54,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 17:00:54,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 17:00:54,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 17:00:54,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 17:00:54,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 17:00:54,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 17:00:54,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 17:00:54,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 17:00:54,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 17:00:54,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 17:00:54,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 17:00:54,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 17:00:54,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 17:00:54,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 17:00:54,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 17:00:54,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 17:00:54,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 17:00:54,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 17:00:54,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 17:00:54,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 17:00:54,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 17:00:54,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 17:00:54,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 17:00:54,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 17:00:54,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 17:00:54,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 17:00:54,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 17:00:54,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 17:00:54,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 17:00:54,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 17:00:54,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 17:00:54,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 17:00:54,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 17:00:54,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 17:00:54,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 17:00:54,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 17:00:54,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 17:00:54,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 17:00:54,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 17:00:54,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 17:00:54,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 17:00:54,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 17:00:54,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 17:00:54,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 17:00:54,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 17:00:54,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 17:00:54,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 17:00:54,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 17:00:54,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 17:00:54,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 17:00:54,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 17:00:54,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 17:00:54,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 17:00:54,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 17:00:54,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 17:00:54,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 17:00:54,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 17:00:54,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 17:00:54,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 17:00:54,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 17:00:54,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 17:00:54,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 17:00:54,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 17:00:54,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 17:00:54,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 17:00:54,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 17:00:54,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 17:00:54,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 17:00:54,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 17:00:54,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 17:00:54,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 17:00:54,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 17:00:54,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 17:00:54,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 17:00:54,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 17:00:54,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 17:00:54,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 17:00:54,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 17:00:54,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 17:00:54,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 17:00:54,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 17:00:54,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 17:00:54,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 17:00:54,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 17:00:54,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 17:00:54,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 17:00:54,785 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:54,786 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-10 17:00:54,786 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-10 17:00:55,124 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 467 ms to try 5000 different iterations.  Found a solution that moves 4727 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17805385942896343. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04072175866329939, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05611836136777174, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5252222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04072175866329939, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17333333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23908333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:55,141 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv428856455:82 , srv1032434426:83 , srv1110155236:83 , srv1223740607:83 , srv582869661:83 , srv646346391:83 , srv675407332:83 , srv1071342175:84 , srv1202360808:84 , srv1273842748:84 , srv1278746631:84 , srv1589659058:84 , srv1619809549:84 , srv1657757764:84 , srv1827503831:84 , srv1967450335:84 , srv1987989601:84 , srv2025832398:84 , srv284755449:84 , srv315422803:84 , srv370409461:84 , srv618626462:84 , srv760181794:84 , srv906061938:84 , srv913645354:84 , srv1098543822:85 , srv1131600258:85 , srv1141145125:85 , srv1147520937:85 , srv11713249:85 , srv1329037891:85 , srv1985614646:85 , srv206161647:85 , srv2126001641:85 , srv270469601:85 , srv273571781:85 , srv358144764:85 , srv555164897:85 , srv719799626:85 , srv831706626:85 , srv1081147243:86 , srv1235269320:86 , srv1456970655:86 , srv1568773295:86 , srv1595606162:86 , srv1742762484:86 , srv195127816:86 , srv1956388418:86 , srv2146471787:86 , srv291943529:86 , srv43116366:86 , srv514260439:86 , srv629735961:86 , srv634831444:86 , srv770899484:86 , srv780068127:86 , srv791148546:86 , srv1018880132:87 , srv1225036245:87 , srv1235067098:87 , srv1472078778:87 , srv1522639362:87 , srv1533009500:87 , srv1589059638:87 , srv1655265206:87 , srv1671627112:87 , srv1724419019:87 , srv1895918247:87 , srv1983114901:87 , srv1991445297:87 , srv231713070:87 , srv316941744:87 , srv712377978:87 , srv794315580:87 , srv845731004:87 , srv1296678761:88 , srv2070195560:88 , srv2109566810:88 , srv262560732:88 , srv392808494:88 , srv629486715:88 , srv1503556082:89 , srv1527574346:89 , srv1563399521:89 , srv1683513782:89 , srv1706005018:89 , srv1784549311:89 , srv1791398501:89 , srv465674193:89 , srv770157472:89 , srv987285979:89 , srv1539973642:90 , srv526931055:90 , srv860708720:90 , srv1515232659:91 , srv1993424891:91 , srv2114399213:91 , srv584240323:94 , srv169412800:98 , srv1764147544:452 }
2024-04-10 17:00:55,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 17:00:55,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1791398501=43, srv358144764=69, srv392808494=71, srv584240323=79, srv634831444=83, srv794315580=93, srv2126001641=59, srv1563399521=27, srv526931055=76, srv1472078778=20, srv1273842748=15, srv1018880132=0, srv1533009500=25, srv1683513782=36, srv1657757764=34, srv1655265206=33, srv1296678761=17, srv195127816=46, srv316941744=68, srv1987989601=51, srv1131600258=6, srv712377978=86, srv514260439=75, srv906061938=97, srv675407332=85, srv1568773295=28, srv262560732=62, srv2114399213=58, srv1993424891=53, srv1225036245=12, srv629486715=81, srv987285979=99, srv1956388418=47, srv1071342175=2, srv1527574346=24, srv760181794=88, srv1967450335=48, srv1589059638=29, srv428856455=72, srv1110155236=5, srv1706005018=38, srv1141145125=7, srv169412800=37, srv1032434426=1, srv1827503831=44, srv780068127=91, srv860708720=96, srv629735961=82, srv1098543822=4, srv273571781=64, srv2109566810=57, srv43116366=73, srv1619809549=32, srv1456970655=19, srv284755449=65, srv770899484=90, srv1539973642=26, srv1589659058=30, srv1784549311=42, srv831706626=94, srv1595606162=31, srv555164897=77, srv231713070=61, srv1278746631=16, srv582869661=78, srv270469601=63, srv646346391=84, srv1522639362=23, srv1983114901=49, srv770157472=89, srv1223740607=11, srv1235067098=13, srv2025832398=54, srv1742762484=40, srv1329037891=18, srv291943529=66, srv1515232659=22, srv1671627112=35, srv845731004=95, srv1895918247=45, srv1985614646=50, srv1503556082=21, srv1202360808=10, srv618626462=80, srv2146471787=60, srv11713249=9, srv1081147243=3, srv1764147544=41, srv465674193=74, srv1147520937=8, srv1991445297=52, srv791148546=92, srv719799626=87, srv2070195560=56, srv1235269320=14, srv370409461=70, srv913645354=98, srv1724419019=39, srv206161647=55, srv315422803=67} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 17:00:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 17:00:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 17:00:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 17:00:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 17:00:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 17:00:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 17:00:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 17:00:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 17:00:55,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 17:00:55,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 17:00:55,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 17:00:55,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 17:00:55,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 17:00:55,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 17:00:55,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 17:00:55,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 17:00:55,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 17:00:55,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 17:00:55,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 17:00:55,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 17:00:55,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 17:00:55,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 17:00:55,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 17:00:55,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 17:00:55,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 17:00:55,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 17:00:55,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 17:00:55,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 17:00:55,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 17:00:55,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 17:00:55,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 17:00:55,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 17:00:55,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 17:00:55,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 17:00:55,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 17:00:55,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 17:00:55,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 17:00:55,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 17:00:55,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 17:00:55,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 17:00:55,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 17:00:55,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 17:00:55,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 17:00:55,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 17:00:55,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 17:00:55,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 17:00:55,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 17:00:55,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 17:00:55,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 17:00:55,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 17:00:55,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 17:00:55,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 17:00:55,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 17:00:55,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 17:00:55,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 17:00:55,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 17:00:55,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 17:00:55,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 17:00:55,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 17:00:55,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 17:00:55,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 17:00:55,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 17:00:55,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 17:00:55,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 17:00:55,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 17:00:55,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 17:00:55,301 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04072175866329939, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05611836136777174, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04072175866329939, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17333333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23908333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:55,301 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-10 17:00:55,302 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17802076026752087, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04072175866329939, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05611836136777174, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04072175866329939, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17333333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23908333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-10 17:00:55,504 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 300 ms to try 5000 different iterations.  Found a solution that moves 2645 regions; Going from a computed imbalance of 0.17802076026752087 to a new imbalance of 0.01370208926300256. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021044552174054678); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0306751561957209, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29388888888888887, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021044552174054678); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041833333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:55,515 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv169412800:75 , srv2114399213:80 , srv584240323:80 , srv1503556082:81 , srv1539973642:81 , srv465674193:81 , srv629486715:81 , srv860708720:81 , srv1527574346:82 , srv1683513782:82 , srv1533009500:83 , srv1563399521:83 , srv1655265206:83 , srv1784549311:83 , srv1993424891:83 , srv316941744:83 , srv1235067098:84 , srv1296678761:84 , srv1329037891:84 , srv1515232659:84 , srv1706005018:84 , srv1983114901:84 , srv1991445297:84 , srv2070195560:84 , srv2109566810:84 , srv770157472:84 , srv794315580:84 , srv845731004:84 , srv987285979:84 , srv1018880132:85 , srv1225036245:85 , srv1522639362:85 , srv1589059638:85 , srv1595606162:85 , srv1671627112:85 , srv1956388418:85 , srv262560732:85 , srv270469601:85 , srv291943529:85 , srv392808494:85 , srv526931055:85 , srv555164897:85 , srv1098543822:86 , srv1110155236:86 , srv1147520937:86 , srv11713249:86 , srv1235269320:86 , srv1472078778:86 , srv1568773295:86 , srv1724419019:86 , srv1742762484:86 , srv1791398501:86 , srv1895918247:86 , srv231713070:86 , srv43116366:86 , srv514260439:86 , srv629735961:86 , srv634831444:86 , srv712377978:86 , srv1071342175:87 , srv1081147243:87 , srv1141145125:87 , srv1202360808:87 , srv1589659058:87 , srv195127816:87 , srv1967450335:87 , srv1985614646:87 , srv273571781:87 , srv284755449:87 , srv315422803:87 , srv358144764:87 , srv618626462:87 , srv791148546:87 , srv1131600258:88 , srv1273842748:88 , srv1456970655:88 , srv1657757764:88 , srv1827503831:88 , srv1987989601:88 , srv2025832398:88 , srv206161647:88 , srv2146471787:88 , srv370409461:88 , srv719799626:88 , srv831706626:88 , srv906061938:88 , srv1032434426:89 , srv1278746631:89 , srv1619809549:89 , srv2126001641:89 , srv760181794:89 , srv770899484:89 , srv428856455:90 , srv582869661:90 , srv780068127:90 , srv913645354:90 , srv1223740607:91 , srv675407332:91 , srv646346391:92 , srv1764147544:509 }
2024-04-10 17:00:55,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 17:00:55,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1791398501=43, srv358144764=69, srv392808494=71, srv584240323=79, srv634831444=83, srv794315580=93, srv2126001641=59, srv1563399521=27, srv526931055=76, srv1472078778=20, srv1273842748=15, srv1018880132=0, srv1533009500=25, srv1683513782=36, srv1657757764=34, srv1655265206=33, srv1296678761=17, srv195127816=46, srv316941744=68, srv1987989601=51, srv1131600258=6, srv712377978=86, srv514260439=75, srv906061938=97, srv675407332=85, srv1568773295=28, srv262560732=62, srv2114399213=58, srv1993424891=53, srv1225036245=12, srv629486715=81, srv987285979=99, srv1956388418=47, srv1071342175=2, srv1527574346=24, srv760181794=88, srv1967450335=48, srv1589059638=29, srv428856455=72, srv1110155236=5, srv1706005018=38, srv1141145125=7, srv169412800=37, srv1032434426=1, srv1827503831=44, srv780068127=91, srv860708720=96, srv629735961=82, srv1098543822=4, srv273571781=64, srv2109566810=57, srv43116366=73, srv1619809549=32, srv1456970655=19, srv284755449=65, srv770899484=90, srv1539973642=26, srv1589659058=30, srv1784549311=42, srv831706626=94, srv1595606162=31, srv555164897=77, srv231713070=61, srv1278746631=16, srv582869661=78, srv270469601=63, srv646346391=84, srv1522639362=23, srv1983114901=49, srv770157472=89, srv1223740607=11, srv1235067098=13, srv2025832398=54, srv1742762484=40, srv1329037891=18, srv291943529=66, srv1515232659=22, srv1671627112=35, srv845731004=95, srv1895918247=45, srv1985614646=50, srv1503556082=21, srv1202360808=10, srv618626462=80, srv2146471787=60, srv11713249=9, srv1081147243=3, srv1764147544=41, srv465674193=74, srv1147520937=8, srv1991445297=52, srv791148546=92, srv719799626=87, srv2070195560=56, srv1235269320=14, srv370409461=70, srv913645354=98, srv1724419019=39, srv206161647=55, srv315422803=67} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 17:00:55,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 17:00:55,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 17:00:55,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 17:00:55,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 17:00:55,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 17:00:55,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 17:00:55,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 17:00:55,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 17:00:55,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 17:00:55,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 17:00:55,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 17:00:55,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 17:00:55,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 17:00:55,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 17:00:55,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 17:00:55,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 17:00:55,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 17:00:55,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 17:00:55,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 17:00:55,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 17:00:55,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 17:00:55,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 17:00:55,687 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021044552174054678); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0306751561957209, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021044552174054678); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041833333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:55,688 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-10 17:00:55,688 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013683568577152003, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021044552174054678); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0306751561957209, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021044552174054678); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041833333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-10 17:00:55,911 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 338 ms to try 5000 different iterations.  Found a solution that moves 953 regions; Going from a computed imbalance of 0.013683568577152003 to a new imbalance of 8.462388678854929E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00387161467279291); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0073702773119008885); MoveCostFunction : (multiplier=7.0, imbalance=0.1058888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00387161467279291); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:55,913 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv780068127:81 , srv1515232659:82 , srv1791398501:82 , srv526931055:82 , srv646346391:83 , srv675407332:83 , srv913645354:83 , srv1131600258:84 , srv1223740607:84 , srv1278746631:84 , srv1456970655:84 , srv195127816:84 , srv1993424891:84 , srv206161647:84 , srv2126001641:84 , srv2146471787:84 , srv231713070:84 , srv262560732:84 , srv584240323:84 , srv719799626:84 , srv760181794:84 , srv770157472:84 , srv770899484:84 , srv1032434426:85 , srv1081147243:85 , srv1141145125:85 , srv1472078778:85 , srv1522639362:85 , srv1619809549:85 , srv169412800:85 , srv1706005018:85 , srv1724419019:85 , srv1742762484:85 , srv1895918247:85 , srv2025832398:85 , srv358144764:85 , srv514260439:85 , srv582869661:85 , srv629735961:85 , srv634831444:85 , srv712377978:85 , srv831706626:85 , srv987285979:85 , srv1018880132:86 , srv1147520937:86 , srv1225036245:86 , srv1235269320:86 , srv1273842748:86 , srv1296678761:86 , srv1563399521:86 , srv1568773295:86 , srv1589059638:86 , srv1589659058:86 , srv1657757764:86 , srv1671627112:86 , srv1784549311:86 , srv1827503831:86 , srv1983114901:86 , srv1987989601:86 , srv1991445297:86 , srv2070195560:86 , srv2109566810:86 , srv2114399213:86 , srv273571781:86 , srv370409461:86 , srv392808494:86 , srv428856455:86 , srv618626462:86 , srv791148546:86 , srv860708720:86 , srv906061938:86 , srv1098543822:87 , srv11713249:87 , srv1202360808:87 , srv1235067098:87 , srv1503556082:87 , srv1527574346:87 , srv1533009500:87 , srv1539973642:87 , srv1595606162:87 , srv1655265206:87 , srv1683513782:87 , srv1956388418:87 , srv1967450335:87 , srv1985614646:87 , srv284755449:87 , srv291943529:87 , srv315422803:87 , srv316941744:87 , srv43116366:87 , srv465674193:87 , srv555164897:87 , srv794315580:87 , srv845731004:87 , srv1071342175:88 , srv1110155236:88 , srv1329037891:88 , srv270469601:88 , srv629486715:88 , srv1764147544:531 }
2024-04-10 17:00:55,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 17:00:55,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1791398501=43, srv358144764=69, srv392808494=71, srv584240323=79, srv634831444=83, srv794315580=93, srv2126001641=59, srv1563399521=27, srv526931055=76, srv1472078778=20, srv1273842748=15, srv1018880132=0, srv1533009500=25, srv1683513782=36, srv1657757764=34, srv1655265206=33, srv1296678761=17, srv195127816=46, srv316941744=68, srv1987989601=51, srv1131600258=6, srv712377978=86, srv514260439=75, srv906061938=97, srv675407332=85, srv1568773295=28, srv262560732=62, srv2114399213=58, srv1993424891=53, srv1225036245=12, srv629486715=81, srv987285979=99, srv1956388418=47, srv1071342175=2, srv1527574346=24, srv760181794=88, srv1967450335=48, srv1589059638=29, srv428856455=72, srv1110155236=5, srv1706005018=38, srv1141145125=7, srv169412800=37, srv1032434426=1, srv1827503831=44, srv780068127=91, srv860708720=96, srv629735961=82, srv1098543822=4, srv273571781=64, srv2109566810=57, srv43116366=73, srv1619809549=32, srv1456970655=19, srv284755449=65, srv770899484=90, srv1539973642=26, srv1589659058=30, srv1784549311=42, srv831706626=94, srv1595606162=31, srv555164897=77, srv231713070=61, srv1278746631=16, srv582869661=78, srv270469601=63, srv646346391=84, srv1522639362=23, srv1983114901=49, srv770157472=89, srv1223740607=11, srv1235067098=13, srv2025832398=54, srv1742762484=40, srv1329037891=18, srv291943529=66, srv1515232659=22, srv1671627112=35, srv845731004=95, srv1895918247=45, srv1985614646=50, srv1503556082=21, srv1202360808=10, srv618626462=80, srv2146471787=60, srv11713249=9, srv1081147243=3, srv1764147544=41, srv465674193=74, srv1147520937=8, srv1991445297=52, srv791148546=92, srv719799626=87, srv2070195560=56, srv1235269320=14, srv370409461=70, srv913645354=98, srv1724419019=39, srv206161647=55, srv315422803=67} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 17:00:55,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 17:00:55,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 17:00:55,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 17:00:55,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 17:00:55,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 17:00:55,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 17:00:55,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 17:00:55,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 17:00:55,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 17:00:55,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 17:00:55,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 17:00:55,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 17:00:55,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 17:00:55,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 17:00:55,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 17:00:55,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 17:00:55,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 17:00:55,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 17:00:55,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 17:00:55,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 17:00:55,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 17:00:55,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 17:00:55,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 17:00:55,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 17:00:55,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 17:00:55,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 17:00:55,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 17:00:55,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 17:00:55,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 17:00:55,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 17:00:55,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 17:00:55,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 17:00:55,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 17:00:55,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 17:00:55,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 17:00:55,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 17:00:55,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 17:00:55,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 17:00:55,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 17:00:55,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 17:00:55,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 17:00:55,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 17:00:55,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 17:00:55,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 17:00:55,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 17:00:55,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 17:00:55,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 17:00:55,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 17:00:55,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 17:00:55,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 17:00:55,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 17:00:55,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 17:00:55,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 17:00:55,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 17:00:55,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 17:00:55,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 17:00:55,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 17:00:55,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 17:00:55,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 17:00:55,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 17:00:55,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 17:00:55,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 17:00:55,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 17:00:55,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 17:00:55,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 17:00:55,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 17:00:55,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 17:00:55,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 17:00:55,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 17:00:55,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 17:00:55,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 17:00:55,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 17:00:55,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 17:00:55,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 17:00:55,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 17:00:55,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 17:00:55,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 17:00:55,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 17:00:55,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 17:00:55,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 17:00:55,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 17:00:55,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 17:00:55,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 17:00:55,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 17:00:55,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 17:00:55,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 17:00:55,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 17:00:55,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 17:00:55,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 17:00:55,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 17:00:56,060 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00387161467279291); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0073702773119008885); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00387161467279291); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:56,060 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-10 17:00:56,060 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.395658192595646E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00387161467279291); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0073702773119008885); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00387161467279291); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-10 17:00:56,436 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 487 ms to try 5000 different iterations.  Found a solution that moves 250 regions; Going from a computed imbalance of 8.395658192595646E-4 to a new imbalance of 2.864935141382591E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.909066927536782E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.027777777777777776, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.909066927536782E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:56,437 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1071342175:83 , srv11713249:83 , srv1235067098:83 , srv1895918247:83 , srv1985614646:83 , srv2109566810:83 , srv392808494:83 , srv43116366:83 , srv770899484:83 , srv1018880132:84 , srv1032434426:84 , srv1081147243:84 , srv1098543822:84 , srv1202360808:84 , srv1235269320:84 , srv1273842748:84 , srv1296678761:84 , srv1472078778:84 , srv1503556082:84 , srv1527574346:84 , srv1533009500:84 , srv1539973642:84 , srv1563399521:84 , srv1568773295:84 , srv1589059638:84 , srv1595606162:84 , srv1619809549:84 , srv1657757764:84 , srv1683513782:84 , srv169412800:84 , srv1706005018:84 , srv1724419019:84 , srv1784549311:84 , srv1827503831:84 , srv1956388418:84 , srv1967450335:84 , srv1993424891:84 , srv2070195560:84 , srv2146471787:84 , srv270469601:84 , srv273571781:84 , srv284755449:84 , srv291943529:84 , srv428856455:84 , srv514260439:84 , srv555164897:84 , srv582869661:84 , srv584240323:84 , srv629486715:84 , srv646346391:84 , srv675407332:84 , srv712377978:84 , srv780068127:84 , srv791148546:84 , srv794315580:84 , srv831706626:84 , srv845731004:84 , srv906061938:84 , srv913645354:84 , srv1110155236:85 , srv1131600258:85 , srv1141145125:85 , srv1147520937:85 , srv1223740607:85 , srv1225036245:85 , srv1278746631:85 , srv1456970655:85 , srv1515232659:85 , srv1589659058:85 , srv1655265206:85 , srv1671627112:85 , srv1742762484:85 , srv1791398501:85 , srv195127816:85 , srv1983114901:85 , srv1987989601:85 , srv1991445297:85 , srv2025832398:85 , srv206161647:85 , srv2114399213:85 , srv2126001641:85 , srv231713070:85 , srv262560732:85 , srv316941744:85 , srv358144764:85 , srv370409461:85 , srv465674193:85 , srv526931055:85 , srv618626462:85 , srv629735961:85 , srv634831444:85 , srv719799626:85 , srv760181794:85 , srv770157472:85 , srv860708720:85 , srv987285979:85 , srv1329037891:86 , srv1522639362:86 , srv315422803:86 , srv1764147544:650 }
2024-04-10 17:00:56,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 17:00:56,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1791398501=43, srv358144764=69, srv392808494=71, srv584240323=79, srv634831444=83, srv794315580=93, srv2126001641=59, srv1563399521=27, srv526931055=76, srv1472078778=20, srv1273842748=15, srv1018880132=0, srv1533009500=25, srv1683513782=36, srv1657757764=34, srv1655265206=33, srv1296678761=17, srv195127816=46, srv316941744=68, srv1987989601=51, srv1131600258=6, srv712377978=86, srv514260439=75, srv906061938=97, srv675407332=85, srv1568773295=28, srv262560732=62, srv2114399213=58, srv1993424891=53, srv1225036245=12, srv629486715=81, srv987285979=99, srv1956388418=47, srv1071342175=2, srv1527574346=24, srv760181794=88, srv1967450335=48, srv1589059638=29, srv428856455=72, srv1110155236=5, srv1706005018=38, srv1141145125=7, srv169412800=37, srv1032434426=1, srv1827503831=44, srv780068127=91, srv860708720=96, srv629735961=82, srv1098543822=4, srv273571781=64, srv2109566810=57, srv43116366=73, srv1619809549=32, srv1456970655=19, srv284755449=65, srv770899484=90, srv1539973642=26, srv1589659058=30, srv1784549311=42, srv831706626=94, srv1595606162=31, srv555164897=77, srv231713070=61, srv1278746631=16, srv582869661=78, srv270469601=63, srv646346391=84, srv1522639362=23, srv1983114901=49, srv770157472=89, srv1223740607=11, srv1235067098=13, srv2025832398=54, srv1742762484=40, srv1329037891=18, srv291943529=66, srv1515232659=22, srv1671627112=35, srv845731004=95, srv1895918247=45, srv1985614646=50, srv1503556082=21, srv1202360808=10, srv618626462=80, srv2146471787=60, srv11713249=9, srv1081147243=3, srv1764147544=41, srv465674193=74, srv1147520937=8, srv1991445297=52, srv791148546=92, srv719799626=87, srv2070195560=56, srv1235269320=14, srv370409461=70, srv913645354=98, srv1724419019=39, srv206161647=55, srv315422803=67} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 17:00:56,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 17:00:56,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 17:00:56,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 17:00:56,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 17:00:56,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 17:00:56,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 17:00:56,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 17:00:56,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 17:00:56,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 17:00:56,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 17:00:56,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 17:00:56,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 17:00:56,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 17:00:56,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 17:00:56,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 17:00:56,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 17:00:56,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 17:00:56,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 17:00:56,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 17:00:56,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 17:00:56,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 17:00:56,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 17:00:56,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 17:00:56,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 17:00:56,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 17:00:56,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 17:00:56,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 17:00:56,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 17:00:56,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 17:00:56,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 17:00:56,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 17:00:56,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 17:00:56,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 17:00:56,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 17:00:56,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 17:00:56,564 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.909066927536782E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.909066927536782E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:56,564 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-10 17:00:56,565 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.689881399883951E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.909066927536782E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.909066927536782E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-10 17:00:56,731 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 253 ms to try 5000 different iterations.  Found a solution that moves 30 regions; Going from a computed imbalance of 2.689881399883951E-5 to a new imbalance of 7.283303786127816E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0033333333333333335); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:56,731 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv315422803:82 , srv1223740607:83 , srv1225036245:83 , srv1329037891:83 , srv1456970655:83 , srv1522639362:83 , srv1671627112:83 , srv1987989601:83 , srv2126001641:83 , srv370409461:83 , srv791148546:83 , srv987285979:83 , srv1018880132:84 , srv1071342175:84 , srv1081147243:84 , srv1098543822:84 , srv1110155236:84 , srv1131600258:84 , srv1141145125:84 , srv1147520937:84 , srv1202360808:84 , srv1235269320:84 , srv1273842748:84 , srv1278746631:84 , srv1296678761:84 , srv1472078778:84 , srv1515232659:84 , srv1527574346:84 , srv1539973642:84 , srv1563399521:84 , srv1568773295:84 , srv1589059638:84 , srv1589659058:84 , srv1595606162:84 , srv1619809549:84 , srv1655265206:84 , srv1657757764:84 , srv1683513782:84 , srv169412800:84 , srv1706005018:84 , srv1724419019:84 , srv1742762484:84 , srv1784549311:84 , srv1791398501:84 , srv1827503831:84 , srv195127816:84 , srv1956388418:84 , srv1967450335:84 , srv1983114901:84 , srv1985614646:84 , srv1991445297:84 , srv1993424891:84 , srv2025832398:84 , srv206161647:84 , srv2070195560:84 , srv2114399213:84 , srv2146471787:84 , srv231713070:84 , srv262560732:84 , srv270469601:84 , srv273571781:84 , srv284755449:84 , srv291943529:84 , srv316941744:84 , srv358144764:84 , srv392808494:84 , srv428856455:84 , srv43116366:84 , srv465674193:84 , srv526931055:84 , srv582869661:84 , srv584240323:84 , srv618626462:84 , srv629735961:84 , srv634831444:84 , srv646346391:84 , srv712377978:84 , srv719799626:84 , srv760181794:84 , srv770157472:84 , srv770899484:84 , srv794315580:84 , srv831706626:84 , srv845731004:84 , srv860708720:84 , srv906061938:84 , srv1032434426:85 , srv11713249:85 , srv1235067098:85 , srv1503556082:85 , srv1533009500:85 , srv1895918247:85 , srv2109566810:85 , srv514260439:85 , srv555164897:85 , srv629486715:85 , srv675407332:85 , srv780068127:85 , srv913645354:85 , srv1764147544:684 }
2024-04-10 17:00:56,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 17:00:56,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1791398501=43, srv358144764=69, srv392808494=71, srv584240323=79, srv634831444=83, srv794315580=93, srv2126001641=59, srv1563399521=27, srv526931055=76, srv1472078778=20, srv1273842748=15, srv1018880132=0, srv1533009500=25, srv1683513782=36, srv1657757764=34, srv1655265206=33, srv1296678761=17, srv195127816=46, srv316941744=68, srv1987989601=51, srv1131600258=6, srv712377978=86, srv514260439=75, srv906061938=97, srv675407332=85, srv1568773295=28, srv262560732=62, srv2114399213=58, srv1993424891=53, srv1225036245=12, srv629486715=81, srv987285979=99, srv1956388418=47, srv1071342175=2, srv1527574346=24, srv760181794=88, srv1967450335=48, srv1589059638=29, srv428856455=72, srv1110155236=5, srv1706005018=38, srv1141145125=7, srv169412800=37, srv1032434426=1, srv1827503831=44, srv780068127=91, srv860708720=96, srv629735961=82, srv1098543822=4, srv273571781=64, srv2109566810=57, srv43116366=73, srv1619809549=32, srv1456970655=19, srv284755449=65, srv770899484=90, srv1539973642=26, srv1589659058=30, srv1784549311=42, srv831706626=94, srv1595606162=31, srv555164897=77, srv231713070=61, srv1278746631=16, srv582869661=78, srv270469601=63, srv646346391=84, srv1522639362=23, srv1983114901=49, srv770157472=89, srv1223740607=11, srv1235067098=13, srv2025832398=54, srv1742762484=40, srv1329037891=18, srv291943529=66, srv1515232659=22, srv1671627112=35, srv845731004=95, srv1895918247=45, srv1985614646=50, srv1503556082=21, srv1202360808=10, srv618626462=80, srv2146471787=60, srv11713249=9, srv1081147243=3, srv1764147544=41, srv465674193=74, srv1147520937=8, srv1991445297=52, srv791148546=92, srv719799626=87, srv2070195560=56, srv1235269320=14, srv370409461=70, srv913645354=98, srv1724419019=39, srv206161647=55, srv315422803=67} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 17:00:56,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 17:00:56,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 17:00:56,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 17:00:56,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 17:00:56,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 17:00:56,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 17:00:56,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 17:00:56,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 17:00:56,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 17:00:56,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 17:00:56,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 17:00:56,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 17:00:56,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 17:00:56,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 17:00:56,835 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:56,835 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv315422803:82 , srv1223740607:83 , srv1225036245:83 , srv1329037891:83 , srv1456970655:83 , srv1522639362:83 , srv1671627112:83 , srv1987989601:83 , srv2126001641:83 , srv370409461:83 , srv791148546:83 , srv987285979:83 , srv1018880132:84 , srv1071342175:84 , srv1081147243:84 , srv1098543822:84 , srv1110155236:84 , srv1131600258:84 , srv1141145125:84 , srv1147520937:84 , srv1202360808:84 , srv1235269320:84 , srv1273842748:84 , srv1278746631:84 , srv1296678761:84 , srv1472078778:84 , srv1515232659:84 , srv1527574346:84 , srv1539973642:84 , srv1563399521:84 , srv1568773295:84 , srv1589059638:84 , srv1589659058:84 , srv1595606162:84 , srv1619809549:84 , srv1655265206:84 , srv1657757764:84 , srv1683513782:84 , srv169412800:84 , srv1706005018:84 , srv1724419019:84 , srv1742762484:84 , srv1784549311:84 , srv1791398501:84 , srv1827503831:84 , srv195127816:84 , srv1956388418:84 , srv1967450335:84 , srv1983114901:84 , srv1985614646:84 , srv1991445297:84 , srv1993424891:84 , srv2025832398:84 , srv206161647:84 , srv2070195560:84 , srv2114399213:84 , srv2146471787:84 , srv231713070:84 , srv262560732:84 , srv270469601:84 , srv273571781:84 , srv284755449:84 , srv291943529:84 , srv316941744:84 , srv358144764:84 , srv392808494:84 , srv428856455:84 , srv43116366:84 , srv465674193:84 , srv526931055:84 , srv582869661:84 , srv584240323:84 , srv618626462:84 , srv629735961:84 , srv634831444:84 , srv646346391:84 , srv712377978:84 , srv719799626:84 , srv760181794:84 , srv770157472:84 , srv770899484:84 , srv794315580:84 , srv831706626:84 , srv845731004:84 , srv860708720:84 , srv906061938:84 , srv1032434426:85 , srv11713249:85 , srv1235067098:85 , srv1503556082:85 , srv1533009500:85 , srv1895918247:85 , srv2109566810:85 , srv514260439:85 , srv555164897:85 , srv629486715:85 , srv675407332:85 , srv780068127:85 , srv913645354:85 , srv1764147544:684 }
2024-04-10 17:00:56,862 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=10695 (was 11547), ProcessCount=192 (was 191) - ProcessCount LEAK? -, AvailableMemoryMB=14409 (was 14903)
2024-04-10 17:00:56,871 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=10695, ProcessCount=191, AvailableMemoryMB=14409
2024-04-10 17:00:56,871 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-10 17:00:56,871 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-04-10 17:00:56,880 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1048433992:84 , srv1051081803:84 , srv1086532963:84 , srv1148164085:84 , srv1239412430:84 , srv128652511:84 , srv1326995604:84 , srv1336747598:84 , srv1490293124:84 , srv1636745319:84 , srv1687551075:84 , srv1774986981:84 , srv1937044367:84 , srv1973448792:84 , srv200735585:84 , srv203650476:84 , srv2130896679:84 , srv2147257926:84 , srv244237131:84 , srv264418733:84 , srv27134095:84 , srv339040374:84 , srv441148545:84 , srv612213556:84 , srv642583892:84 , srv644786545:84 , srv782729923:84 , srv789735331:84 , srv944441288:84 , srv1722556524:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-04-10 17:00:56,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 17:00:56,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv789735331=28, srv1336747598=7, srv27134095=21, srv244237131=19, srv1973448792=14, srv2130896679=17, srv200735585=15, srv1086532963=2, srv1048433992=0, srv128652511=5, srv1937044367=13, srv203650476=16, srv264418733=20, srv612213556=24, srv1239412430=4, srv1490293124=8, srv1148164085=3, srv1051081803=1, srv1774986981=12, srv944441288=29, srv1687551075=10, srv441148545=23, srv782729923=27, srv1722556524=11, srv644786545=26, srv339040374=22, srv1326995604=6, srv2147257926=18, srv1636745319=9, srv642583892=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 17:00:56,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 17:00:56,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 17:00:56,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 17:00:56,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-10 17:00:56,905 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0592095384930404, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:00:56,905 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912379610586702, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0592095384930404, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-04-10 17:02:56,894 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9486993 different iterations.  Found a solution that moves 1834 regions; Going from a computed imbalance of 0.9912379610586702 to a new imbalance of 4.280647492113412E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6792592592592592, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:02:56,899 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1048433992:90 , srv1051081803:90 , srv1086532963:90 , srv1148164085:90 , srv1239412430:90 , srv128652511:90 , srv1326995604:90 , srv1336747598:90 , srv1490293124:90 , srv1636745319:90 , srv1687551075:90 , srv1722556524:90 , srv1774986981:90 , srv1937044367:90 , srv1973448792:90 , srv200735585:90 , srv203650476:90 , srv2130896679:90 , srv2147257926:90 , srv244237131:90 , srv264418733:90 , srv27134095:90 , srv339040374:90 , srv441148545:90 , srv612213556:90 , srv642583892:90 , srv644786545:90 , srv782729923:90 , srv789735331:90 , srv944441288:90 }
2024-04-10 17:02:56,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 17:02:56,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv789735331=28, srv1336747598=7, srv27134095=21, srv244237131=19, srv1973448792=14, srv2130896679=17, srv200735585=15, srv1086532963=2, srv1048433992=0, srv128652511=5, srv1937044367=13, srv203650476=16, srv264418733=20, srv612213556=24, srv1239412430=4, srv1490293124=8, srv1148164085=3, srv1051081803=1, srv1774986981=12, srv944441288=29, srv1687551075=10, srv441148545=23, srv782729923=27, srv1722556524=11, srv644786545=26, srv339040374=22, srv1326995604=6, srv2147257926=18, srv1636745319=9, srv642583892=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 17:02:56,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 17:02:56,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 17:02:56,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 17:02:56,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 17:02:56,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 17:02:56,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 17:02:56,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 17:02:56,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 17:02:56,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-10 17:02:56,915 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 17:02:56,916 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1048433992:90 , srv1051081803:90 , srv1086532963:90 , srv1148164085:90 , srv1239412430:90 , srv128652511:90 , srv1326995604:90 , srv1336747598:90 , srv1490293124:90 , srv1636745319:90 , srv1687551075:90 , srv1722556524:90 , srv1774986981:90 , srv1937044367:90 , srv1973448792:90 , srv200735585:90 , srv203650476:90 , srv2130896679:90 , srv2147257926:90 , srv244237131:90 , srv264418733:90 , srv27134095:90 , srv339040374:90 , srv441148545:90 , srv612213556:90 , srv642583892:90 , srv644786545:90 , srv782729923:90 , srv789735331:90 , srv944441288:90 }
2024-04-10 17:02:56,929 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1580 (was 10695), ProcessCount=187 (was 191), AvailableMemoryMB=14490 (was 14409) - AvailableMemoryMB LEAK? -