2024-04-13 22:53:05,470 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-04-13 22:53:05,806 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-04-13 22:53:05,920 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-04-13 22:53:05,989 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-13 22:53:05,997 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-13 22:53:06,012 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=285, ProcessCount=170, AvailableMemoryMB=8824
2024-04-13 22:53:06,015 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-13 22:53:06,015 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-13 22:53:06,146 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1018241693:84 , srv1045009753:84 , srv107526439:84 , srv1076937052:84 , srv1084296280:84 , srv1087842375:84 , srv1108871989:84 , srv1109728096:84 , srv1124423441:84 , srv1148938881:84 , srv1188672681:84 , srv1274615449:84 , srv1293742068:84 , srv1329244192:84 , srv1330584156:84 , srv1336981486:84 , srv1365765018:84 , srv1380480853:84 , srv1416015879:84 , srv1421664875:84 , srv1432087377:84 , srv1454523552:84 , srv1459026087:84 , srv1460116671:84 , srv1471479524:84 , srv1473462474:84 , srv1510956296:84 , srv1576095731:84 , srv1579717386:84 , srv1581110249:84 , srv158852500:84 , srv1597181299:84 , srv1606939294:84 , srv1666652875:84 , srv167025777:84 , srv1708254125:84 , srv1742813294:84 , srv1758452109:84 , srv1822784994:84 , srv1845168347:84 , srv1854628815:84 , srv1877917428:84 , srv189982352:84 , srv1912567534:84 , srv1952557611:84 , srv1983792683:84 , srv2000186461:84 , srv2009908542:84 , srv2017899360:84 , srv2026000167:84 , srv2037837768:84 , srv2058662995:84 , srv2117673955:84 , srv2122787615:84 , srv2139179669:84 , srv216008860:84 , srv233444953:84 , srv257460250:84 , srv263700835:84 , srv316401114:84 , srv323732350:84 , srv330940473:84 , srv339580990:84 , srv360163538:84 , srv400101786:84 , srv43763272:84 , srv458982904:84 , srv46934440:84 , srv492323684:84 , srv50305666:84 , srv526535763:84 , srv58490646:84 , srv590454074:84 , srv596176104:84 , srv603666649:84 , srv644356081:84 , srv66252324:84 , srv688346873:84 , srv690402850:84 , srv694328535:84 , srv707937541:84 , srv719951464:84 , srv720801613:84 , srv735091052:84 , srv737077315:84 , srv739076226:84 , srv765900903:84 , srv769299798:84 , srv794582146:84 , srv810323536:84 , srv813943996:84 , srv855456242:84 , srv890979605:84 , srv892303240:84 , srv907918629:84 , srv926398873:84 , srv953797549:84 , srv954490018:84 , srv962223200:84 , srv1374165157:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-04-13 22:53:06,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-13 22:53:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1845168347=40, srv316401114=60, srv1045009753=1, srv1510956296=27, srv1365765018=16, srv216008860=56, srv263700835=59, srv189982352=43, srv1084296280=4, srv1380480853=18, srv2017899360=49, srv926398873=96, srv1460116671=24, srv1076937052=3, srv1606939294=33, srv892303240=94, srv107526439=2, srv1473462474=26, srv360163538=64, srv1822784994=39, srv1148938881=9, srv400101786=65, srv953797549=97, srv2117673955=53, srv644356081=76, srv1330584156=14, srv167025777=35, srv694328535=80, srv719951464=82, srv1293742068=12, srv526535763=71, srv1854628815=41, srv1454523552=22, srv158852500=31, srv765900903=87, srv735091052=84, srv1597181299=32, srv1329244192=13, srv257460250=58, srv690402850=79, srv1877917428=42, srv330940473=62, srv1087842375=5, srv458982904=67, srv46934440=68, srv1018241693=0, srv855456242=92, srv720801613=83, srv1432087377=21, srv737077315=85, srv2026000167=50, srv58490646=72, srv43763272=66, srv1336981486=15, srv2139179669=55, srv688346873=78, srv2122787615=54, srv1579717386=29, srv1758452109=38, srv233444953=57, srv2009908542=48, srv769299798=88, srv1952557611=45, srv1708254125=36, srv794582146=89, srv1109728096=7, srv2000186461=47, srv492323684=69, srv2037837768=51, srv323732350=61, srv1416015879=19, srv1576095731=28, srv1471479524=25, srv1188672681=10, srv596176104=74, srv1742813294=37, srv907918629=95, srv954490018=98, srv1374165157=17, srv810323536=90, srv1459026087=23, srv813943996=91, srv1108871989=6, srv707937541=81, srv890979605=93, srv739076226=86, srv1983792683=46, srv339580990=63, srv50305666=70, srv603666649=75, srv66252324=77, srv962223200=99, srv1421664875=20, srv1912567534=44, srv1666652875=34, srv1124423441=8, srv1581110249=30, srv1274615449=11, srv590454074=73, srv2058662995=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-13 22:53:06,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-13 22:53:06,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-13 22:53:06,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-13 22:53:06,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-13 22:53:06,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-13 22:53:06,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-13 22:53:06,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-13 22:53:06,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-13 22:53:06,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-13 22:53:06,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-13 22:53:06,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-13 22:53:06,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-13 22:53:06,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-13 22:53:06,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-13 22:53:06,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-13 22:53:06,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-13 22:53:06,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-13 22:53:06,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-13 22:53:06,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-13 22:53:06,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-13 22:53:06,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-13 22:53:06,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-13 22:53:06,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-13 22:53:06,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-13 22:53:06,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-13 22:53:06,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-13 22:53:06,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-13 22:53:06,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-13 22:53:06,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-13 22:53:06,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-13 22:53:06,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-13 22:53:06,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-13 22:53:06,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-13 22:53:06,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-13 22:53:06,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-13 22:53:06,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-13 22:53:06,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-13 22:53:06,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-13 22:53:06,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-13 22:53:06,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-13 22:53:06,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-13 22:53:06,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-13 22:53:06,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-13 22:53:06,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-13 22:53:06,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-13 22:53:06,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-13 22:53:06,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-13 22:53:06,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-13 22:53:06,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-13 22:53:06,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-13 22:53:06,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-13 22:53:06,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-13 22:53:06,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-13 22:53:06,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-13 22:53:06,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-13 22:53:06,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-13 22:53:06,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-13 22:53:06,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-13 22:53:06,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-13 22:53:06,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-13 22:53:06,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-13 22:53:06,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-13 22:53:06,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-13 22:53:06,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-13 22:53:06,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-13 22:53:06,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-13 22:53:06,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-13 22:53:06,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-13 22:53:06,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-13 22:53:06,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-13 22:53:06,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-13 22:53:06,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-13 22:53:06,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-13 22:53:06,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-13 22:53:06,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-13 22:53:06,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-13 22:53:06,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-13 22:53:06,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-13 22:53:06,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-13 22:53:06,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-13 22:53:06,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-13 22:53:06,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-13 22:53:06,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-13 22:53:06,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-13 22:53:06,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-13 22:53:06,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-13 22:53:06,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-13 22:53:06,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-13 22:53:06,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-13 22:53:06,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-13 22:53:06,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-13 22:53:06,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-13 22:53:06,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-13 22:53:06,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-13 22:53:06,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-13 22:53:06,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-13 22:53:06,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-13 22:53:06,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-13 22:53:06,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-13 22:53:06,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-13 22:53:06,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-13 22:53:06,578 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:06,578 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-13 22:53:06,579 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-13 22:53:06,963 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 654 ms to try 5000 different iterations.  Found a solution that moves 4653 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18796684066282962. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04232222783536739, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05839607015973772, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.517, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04232222783536739, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18316666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25066666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:06,983 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1459026087:82 , srv1471479524:82 , srv2117673955:82 , srv954490018:82 , srv1293742068:83 , srv1416015879:83 , srv167025777:83 , srv2009908542:83 , srv2026000167:83 , srv2139179669:83 , srv216008860:83 , srv316401114:83 , srv360163538:83 , srv707937541:83 , srv892303240:83 , srv926398873:83 , srv1579717386:84 , srv1666652875:84 , srv1758452109:84 , srv1854628815:84 , srv2000186461:84 , srv263700835:84 , srv50305666:84 , srv526535763:84 , srv690402850:84 , srv694328535:84 , srv720801613:84 , srv1045009753:85 , srv1124423441:85 , srv1188672681:85 , srv1330584156:85 , srv1336981486:85 , srv1421664875:85 , srv1845168347:85 , srv1912567534:85 , srv2037837768:85 , srv2122787615:85 , srv58490646:85 , srv644356081:85 , srv735091052:85 , srv765900903:85 , srv107526439:86 , srv1365765018:86 , srv1380480853:86 , srv1460116671:86 , srv1473462474:86 , srv1576095731:86 , srv1597181299:86 , srv1606939294:86 , srv1708254125:86 , srv1877917428:86 , srv1952557611:86 , srv2058662995:86 , srv257460250:86 , srv339580990:86 , srv492323684:86 , srv603666649:86 , srv688346873:86 , srv739076226:86 , srv813943996:86 , srv1076937052:87 , srv1274615449:87 , srv1510956296:87 , srv1581110249:87 , srv1983792683:87 , srv2017899360:87 , srv323732350:87 , srv330940473:87 , srv400101786:87 , srv43763272:87 , srv458982904:87 , srv596176104:87 , srv737077315:87 , srv1084296280:88 , srv1148938881:88 , srv1329244192:88 , srv1454523552:88 , srv1742813294:88 , srv590454074:88 , srv810323536:88 , srv890979605:88 , srv953797549:88 , srv962223200:88 , srv1018241693:89 , srv158852500:89 , srv1822784994:89 , srv189982352:89 , srv46934440:89 , srv66252324:89 , srv719951464:89 , srv769299798:89 , srv794582146:89 , srv855456242:89 , srv907918629:89 , srv1108871989:90 , srv1109728096:90 , srv233444953:94 , srv1432087377:96 , srv1087842375:102 , srv1374165157:466 }
2024-04-13 22:53:06,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-13 22:53:06,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1845168347=40, srv316401114=60, srv1045009753=1, srv1510956296=27, srv1365765018=16, srv216008860=56, srv263700835=59, srv189982352=43, srv1084296280=4, srv1380480853=18, srv2017899360=49, srv926398873=96, srv1460116671=24, srv1076937052=3, srv1606939294=33, srv892303240=94, srv107526439=2, srv1473462474=26, srv360163538=64, srv1822784994=39, srv1148938881=9, srv400101786=65, srv953797549=97, srv2117673955=53, srv644356081=76, srv1330584156=14, srv167025777=35, srv694328535=80, srv719951464=82, srv1293742068=12, srv526535763=71, srv1854628815=41, srv1454523552=22, srv158852500=31, srv765900903=87, srv735091052=84, srv1597181299=32, srv1329244192=13, srv257460250=58, srv690402850=79, srv1877917428=42, srv330940473=62, srv1087842375=5, srv458982904=67, srv46934440=68, srv1018241693=0, srv855456242=92, srv720801613=83, srv1432087377=21, srv737077315=85, srv2026000167=50, srv58490646=72, srv43763272=66, srv1336981486=15, srv2139179669=55, srv688346873=78, srv2122787615=54, srv1579717386=29, srv1758452109=38, srv233444953=57, srv2009908542=48, srv769299798=88, srv1952557611=45, srv1708254125=36, srv794582146=89, srv1109728096=7, srv2000186461=47, srv492323684=69, srv2037837768=51, srv323732350=61, srv1416015879=19, srv1576095731=28, srv1471479524=25, srv1188672681=10, srv596176104=74, srv1742813294=37, srv907918629=95, srv954490018=98, srv1374165157=17, srv810323536=90, srv1459026087=23, srv813943996=91, srv1108871989=6, srv707937541=81, srv890979605=93, srv739076226=86, srv1983792683=46, srv339580990=63, srv50305666=70, srv603666649=75, srv66252324=77, srv962223200=99, srv1421664875=20, srv1912567534=44, srv1666652875=34, srv1124423441=8, srv1581110249=30, srv1274615449=11, srv590454074=73, srv2058662995=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-13 22:53:07,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-13 22:53:07,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-13 22:53:07,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-13 22:53:07,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-13 22:53:07,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-13 22:53:07,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-13 22:53:07,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-13 22:53:07,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-13 22:53:07,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-13 22:53:07,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-13 22:53:07,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-13 22:53:07,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-13 22:53:07,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-13 22:53:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-13 22:53:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-13 22:53:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-13 22:53:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-13 22:53:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-13 22:53:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-13 22:53:07,221 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04232222783536739, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05839607015973772, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04232222783536739, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18316666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25066666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:07,222 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-13 22:53:07,222 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1879342596604619, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04232222783536739, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05839607015973772, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04232222783536739, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18316666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25066666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-13 22:53:07,528 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 483 ms to try 5000 different iterations.  Found a solution that moves 2636 regions; Going from a computed imbalance of 0.1879342596604619 to a new imbalance of 0.015286214620131548. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022725501863918283); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03581032461520023, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29288888888888887, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022725501863918283); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04658333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:07,542 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1087842375:71 , srv1432087377:74 , srv233444953:79 , srv1109728096:81 , srv158852500:81 , srv769299798:81 , srv1148938881:82 , srv1822784994:82 , srv323732350:82 , srv719951464:82 , srv1108871989:83 , srv1329244192:83 , srv1581110249:83 , srv189982352:83 , srv2017899360:83 , srv66252324:83 , srv890979605:83 , srv907918629:83 , srv953797549:83 , srv1084296280:84 , srv1274615449:84 , srv1454523552:84 , srv1473462474:84 , srv1708254125:84 , srv1742813294:84 , srv46934440:84 , srv596176104:84 , srv739076226:84 , srv794582146:84 , srv855456242:84 , srv962223200:84 , srv1606939294:85 , srv1912567534:85 , srv1952557611:85 , srv1983792683:85 , srv330940473:85 , srv339580990:85 , srv400101786:85 , srv492323684:85 , srv590454074:85 , srv810323536:85 , srv107526439:86 , srv1076937052:86 , srv1124423441:86 , srv1336981486:86 , srv1380480853:86 , srv1576095731:86 , srv1597181299:86 , srv2037837768:86 , srv2058662995:86 , srv257460250:86 , srv43763272:86 , srv458982904:86 , srv603666649:86 , srv688346873:86 , srv690402850:86 , srv694328535:86 , srv813943996:86 , srv1018241693:87 , srv1188672681:87 , srv1365765018:87 , srv1421664875:87 , srv1460116671:87 , srv1510956296:87 , srv1666652875:87 , srv1758452109:87 , srv1845168347:87 , srv50305666:87 , srv58490646:87 , srv735091052:87 , srv1045009753:88 , srv1330584156:88 , srv1416015879:88 , srv1471479524:88 , srv1579717386:88 , srv1854628815:88 , srv1877917428:88 , srv2000186461:88 , srv2026000167:88 , srv216008860:88 , srv263700835:88 , srv316401114:88 , srv360163538:88 , srv526535763:88 , srv2122787615:89 , srv707937541:89 , srv720801613:89 , srv737077315:89 , srv926398873:89 , srv954490018:89 , srv167025777:90 , srv2117673955:90 , srv644356081:90 , srv765900903:90 , srv1293742068:91 , srv1459026087:91 , srv892303240:91 , srv2009908542:92 , srv2139179669:93 , srv1374165157:510 }
2024-04-13 22:53:07,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-13 22:53:07,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1845168347=40, srv316401114=60, srv1045009753=1, srv1510956296=27, srv1365765018=16, srv216008860=56, srv263700835=59, srv189982352=43, srv1084296280=4, srv1380480853=18, srv2017899360=49, srv926398873=96, srv1460116671=24, srv1076937052=3, srv1606939294=33, srv892303240=94, srv107526439=2, srv1473462474=26, srv360163538=64, srv1822784994=39, srv1148938881=9, srv400101786=65, srv953797549=97, srv2117673955=53, srv644356081=76, srv1330584156=14, srv167025777=35, srv694328535=80, srv719951464=82, srv1293742068=12, srv526535763=71, srv1854628815=41, srv1454523552=22, srv158852500=31, srv765900903=87, srv735091052=84, srv1597181299=32, srv1329244192=13, srv257460250=58, srv690402850=79, srv1877917428=42, srv330940473=62, srv1087842375=5, srv458982904=67, srv46934440=68, srv1018241693=0, srv855456242=92, srv720801613=83, srv1432087377=21, srv737077315=85, srv2026000167=50, srv58490646=72, srv43763272=66, srv1336981486=15, srv2139179669=55, srv688346873=78, srv2122787615=54, srv1579717386=29, srv1758452109=38, srv233444953=57, srv2009908542=48, srv769299798=88, srv1952557611=45, srv1708254125=36, srv794582146=89, srv1109728096=7, srv2000186461=47, srv492323684=69, srv2037837768=51, srv323732350=61, srv1416015879=19, srv1576095731=28, srv1471479524=25, srv1188672681=10, srv596176104=74, srv1742813294=37, srv907918629=95, srv954490018=98, srv1374165157=17, srv810323536=90, srv1459026087=23, srv813943996=91, srv1108871989=6, srv707937541=81, srv890979605=93, srv739076226=86, srv1983792683=46, srv339580990=63, srv50305666=70, srv603666649=75, srv66252324=77, srv962223200=99, srv1421664875=20, srv1912567534=44, srv1666652875=34, srv1124423441=8, srv1581110249=30, srv1274615449=11, srv590454074=73, srv2058662995=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-13 22:53:07,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-13 22:53:07,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-13 22:53:07,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-13 22:53:07,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-13 22:53:07,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-13 22:53:07,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-13 22:53:07,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-13 22:53:07,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-13 22:53:07,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-13 22:53:07,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-13 22:53:07,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-13 22:53:07,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-13 22:53:07,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-13 22:53:07,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-13 22:53:07,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-13 22:53:07,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-13 22:53:07,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-13 22:53:07,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-13 22:53:07,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-13 22:53:07,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-13 22:53:07,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-13 22:53:07,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-13 22:53:07,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-13 22:53:07,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-13 22:53:07,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-13 22:53:07,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-13 22:53:07,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-13 22:53:07,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-13 22:53:07,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-13 22:53:07,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-13 22:53:07,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-13 22:53:07,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-13 22:53:07,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-13 22:53:07,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-13 22:53:07,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-13 22:53:07,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-13 22:53:07,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-13 22:53:07,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-13 22:53:07,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-13 22:53:07,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-13 22:53:07,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-13 22:53:07,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-13 22:53:07,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-13 22:53:07,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-13 22:53:07,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-13 22:53:07,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-13 22:53:07,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-13 22:53:07,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-13 22:53:07,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-13 22:53:07,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-13 22:53:07,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-13 22:53:07,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-13 22:53:07,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-13 22:53:07,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-13 22:53:07,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-13 22:53:07,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-13 22:53:07,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-13 22:53:07,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-13 22:53:07,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-13 22:53:07,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-13 22:53:07,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-13 22:53:07,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-13 22:53:07,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-13 22:53:07,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-13 22:53:07,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-13 22:53:07,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-13 22:53:07,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-13 22:53:07,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-13 22:53:07,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-13 22:53:07,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-13 22:53:07,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-13 22:53:07,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-13 22:53:07,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-13 22:53:07,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-13 22:53:07,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-13 22:53:07,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-13 22:53:07,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-13 22:53:07,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-13 22:53:07,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-13 22:53:07,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-13 22:53:07,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-13 22:53:07,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-13 22:53:07,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-13 22:53:07,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-13 22:53:07,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-13 22:53:07,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-13 22:53:07,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-13 22:53:07,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-13 22:53:07,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-13 22:53:07,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-13 22:53:07,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-13 22:53:07,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-13 22:53:07,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-13 22:53:07,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-13 22:53:07,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-13 22:53:07,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-13 22:53:07,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-13 22:53:07,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-13 22:53:07,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-13 22:53:07,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-13 22:53:07,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-13 22:53:07,743 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022725501863918283); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03581032461520023, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022725501863918283); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04658333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:07,743 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-13 22:53:07,744 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01526775695362793, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022725501863918283); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03581032461520023, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022725501863918283); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04658333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-13 22:53:08,016 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 402 ms to try 5000 different iterations.  Found a solution that moves 1046 regions; Going from a computed imbalance of 0.01526775695362793 to a new imbalance of 7.845760581760761E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007338059219328525); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008151208485588693); MoveCostFunction : (multiplier=7.0, imbalance=0.11622222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007338059219328525); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:08,019 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1018241693:81 , srv2139179669:81 , srv765900903:81 , srv2009908542:82 , srv644356081:82 , srv737077315:82 , srv1877917428:83 , srv1045009753:84 , srv1076937052:84 , srv1293742068:84 , srv1330584156:84 , srv1460116671:84 , srv1510956296:84 , srv167025777:84 , srv1742813294:84 , srv2058662995:84 , srv2122787615:84 , srv233444953:84 , srv263700835:84 , srv46934440:84 , srv58490646:84 , srv590454074:84 , srv794582146:84 , srv810323536:84 , srv855456242:84 , srv892303240:84 , srv1087842375:85 , srv1108871989:85 , srv1336981486:85 , srv1365765018:85 , srv1380480853:85 , srv1421664875:85 , srv1454523552:85 , srv1459026087:85 , srv1576095731:85 , srv1579717386:85 , srv1597181299:85 , srv1845168347:85 , srv189982352:85 , srv2000186461:85 , srv2037837768:85 , srv257460250:85 , srv360163538:85 , srv400101786:85 , srv43763272:85 , srv458982904:85 , srv526535763:85 , srv603666649:85 , srv66252324:85 , srv720801613:85 , srv735091052:85 , srv907918629:85 , srv926398873:85 , srv107526439:86 , srv1084296280:86 , srv1109728096:86 , srv1124423441:86 , srv1188672681:86 , srv1432087377:86 , srv1708254125:86 , srv1854628815:86 , srv1983792683:86 , srv2117673955:86 , srv339580990:86 , srv492323684:86 , srv688346873:86 , srv719951464:86 , srv813943996:86 , srv890979605:86 , srv953797549:86 , srv954490018:86 , srv962223200:86 , srv1148938881:87 , srv1274615449:87 , srv1329244192:87 , srv1416015879:87 , srv1471479524:87 , srv1581110249:87 , srv158852500:87 , srv1606939294:87 , srv1666652875:87 , srv1758452109:87 , srv1822784994:87 , srv1912567534:87 , srv216008860:87 , srv316401114:87 , srv330940473:87 , srv50305666:87 , srv596176104:87 , srv690402850:87 , srv707937541:87 , srv769299798:87 , srv1473462474:88 , srv1952557611:88 , srv2017899360:88 , srv2026000167:88 , srv323732350:88 , srv694328535:88 , srv739076226:88 , srv1374165157:547 }
2024-04-13 22:53:08,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-13 22:53:08,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1845168347=40, srv316401114=60, srv1045009753=1, srv1510956296=27, srv1365765018=16, srv216008860=56, srv263700835=59, srv189982352=43, srv1084296280=4, srv1380480853=18, srv2017899360=49, srv926398873=96, srv1460116671=24, srv1076937052=3, srv1606939294=33, srv892303240=94, srv107526439=2, srv1473462474=26, srv360163538=64, srv1822784994=39, srv1148938881=9, srv400101786=65, srv953797549=97, srv2117673955=53, srv644356081=76, srv1330584156=14, srv167025777=35, srv694328535=80, srv719951464=82, srv1293742068=12, srv526535763=71, srv1854628815=41, srv1454523552=22, srv158852500=31, srv765900903=87, srv735091052=84, srv1597181299=32, srv1329244192=13, srv257460250=58, srv690402850=79, srv1877917428=42, srv330940473=62, srv1087842375=5, srv458982904=67, srv46934440=68, srv1018241693=0, srv855456242=92, srv720801613=83, srv1432087377=21, srv737077315=85, srv2026000167=50, srv58490646=72, srv43763272=66, srv1336981486=15, srv2139179669=55, srv688346873=78, srv2122787615=54, srv1579717386=29, srv1758452109=38, srv233444953=57, srv2009908542=48, srv769299798=88, srv1952557611=45, srv1708254125=36, srv794582146=89, srv1109728096=7, srv2000186461=47, srv492323684=69, srv2037837768=51, srv323732350=61, srv1416015879=19, srv1576095731=28, srv1471479524=25, srv1188672681=10, srv596176104=74, srv1742813294=37, srv907918629=95, srv954490018=98, srv1374165157=17, srv810323536=90, srv1459026087=23, srv813943996=91, srv1108871989=6, srv707937541=81, srv890979605=93, srv739076226=86, srv1983792683=46, srv339580990=63, srv50305666=70, srv603666649=75, srv66252324=77, srv962223200=99, srv1421664875=20, srv1912567534=44, srv1666652875=34, srv1124423441=8, srv1581110249=30, srv1274615449=11, srv590454074=73, srv2058662995=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-13 22:53:08,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-13 22:53:08,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-13 22:53:08,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-13 22:53:08,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-13 22:53:08,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-13 22:53:08,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-13 22:53:08,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-13 22:53:08,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-13 22:53:08,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-13 22:53:08,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-13 22:53:08,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-13 22:53:08,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-13 22:53:08,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-13 22:53:08,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-13 22:53:08,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-13 22:53:08,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-13 22:53:08,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-13 22:53:08,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-13 22:53:08,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-13 22:53:08,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-13 22:53:08,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-13 22:53:08,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-13 22:53:08,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-13 22:53:08,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-13 22:53:08,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-13 22:53:08,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-13 22:53:08,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-13 22:53:08,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-13 22:53:08,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-13 22:53:08,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-13 22:53:08,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-13 22:53:08,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-13 22:53:08,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-13 22:53:08,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-13 22:53:08,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-13 22:53:08,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-13 22:53:08,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-13 22:53:08,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-13 22:53:08,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-13 22:53:08,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-13 22:53:08,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-13 22:53:08,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-13 22:53:08,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-13 22:53:08,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-13 22:53:08,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-13 22:53:08,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-13 22:53:08,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-13 22:53:08,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-13 22:53:08,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-13 22:53:08,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-13 22:53:08,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-13 22:53:08,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-13 22:53:08,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-13 22:53:08,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-13 22:53:08,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-13 22:53:08,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-13 22:53:08,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-13 22:53:08,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-13 22:53:08,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-13 22:53:08,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-13 22:53:08,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-13 22:53:08,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-13 22:53:08,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-13 22:53:08,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-13 22:53:08,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-13 22:53:08,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-13 22:53:08,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-13 22:53:08,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-13 22:53:08,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-13 22:53:08,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-13 22:53:08,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-13 22:53:08,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-13 22:53:08,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-13 22:53:08,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-13 22:53:08,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-13 22:53:08,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-13 22:53:08,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-13 22:53:08,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-13 22:53:08,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-13 22:53:08,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-13 22:53:08,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-13 22:53:08,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-13 22:53:08,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-13 22:53:08,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-13 22:53:08,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-13 22:53:08,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-13 22:53:08,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-13 22:53:08,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-13 22:53:08,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-13 22:53:08,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-13 22:53:08,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-13 22:53:08,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-13 22:53:08,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-13 22:53:08,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-13 22:53:08,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-13 22:53:08,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-13 22:53:08,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-13 22:53:08,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-13 22:53:08,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-13 22:53:08,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-13 22:53:08,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-13 22:53:08,193 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007338059219328525); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008151208485588693); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007338059219328525); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:08,193 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-13 22:53:08,193 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.77251809631773E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007338059219328525); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008151208485588693); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007338059219328525); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-13 22:53:08,576 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 505 ms to try 5000 different iterations.  Found a solution that moves 258 regions; Going from a computed imbalance of 7.77251809631773E-4 to a new imbalance of 4.350476277928387E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.028666666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:08,577 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1952557611:83 , srv2026000167:83 , srv2117673955:83 , srv330940473:83 , srv707937541:83 , srv962223200:83 , srv107526439:84 , srv1084296280:84 , srv1087842375:84 , srv1108871989:84 , srv1274615449:84 , srv1293742068:84 , srv1329244192:84 , srv1365765018:84 , srv1416015879:84 , srv1459026087:84 , srv1473462474:84 , srv1510956296:84 , srv1606939294:84 , srv1666652875:84 , srv1758452109:84 , srv1822784994:84 , srv1854628815:84 , srv1983792683:84 , srv2009908542:84 , srv2017899360:84 , srv2122787615:84 , srv216008860:84 , srv316401114:84 , srv43763272:84 , srv458982904:84 , srv50305666:84 , srv596176104:84 , srv688346873:84 , srv694328535:84 , srv720801613:84 , srv737077315:84 , srv813943996:84 , srv890979605:84 , srv892303240:84 , srv953797549:84 , srv1018241693:85 , srv1045009753:85 , srv1076937052:85 , srv1109728096:85 , srv1124423441:85 , srv1148938881:85 , srv1188672681:85 , srv1380480853:85 , srv1421664875:85 , srv1454523552:85 , srv1460116671:85 , srv1471479524:85 , srv1576095731:85 , srv1579717386:85 , srv158852500:85 , srv1597181299:85 , srv167025777:85 , srv1742813294:85 , srv189982352:85 , srv1912567534:85 , srv2000186461:85 , srv2139179669:85 , srv233444953:85 , srv257460250:85 , srv263700835:85 , srv323732350:85 , srv339580990:85 , srv400101786:85 , srv46934440:85 , srv492323684:85 , srv526535763:85 , srv58490646:85 , srv590454074:85 , srv603666649:85 , srv644356081:85 , srv66252324:85 , srv690402850:85 , srv719951464:85 , srv735091052:85 , srv739076226:85 , srv769299798:85 , srv794582146:85 , srv855456242:85 , srv907918629:85 , srv954490018:85 , srv1330584156:86 , srv1336981486:86 , srv1432087377:86 , srv1581110249:86 , srv1708254125:86 , srv1845168347:86 , srv1877917428:86 , srv2037837768:86 , srv2058662995:86 , srv360163538:86 , srv765900903:86 , srv810323536:86 , srv926398873:86 , srv1374165157:619 }
2024-04-13 22:53:08,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-13 22:53:08,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1845168347=40, srv316401114=60, srv1045009753=1, srv1510956296=27, srv1365765018=16, srv216008860=56, srv263700835=59, srv189982352=43, srv1084296280=4, srv1380480853=18, srv2017899360=49, srv926398873=96, srv1460116671=24, srv1076937052=3, srv1606939294=33, srv892303240=94, srv107526439=2, srv1473462474=26, srv360163538=64, srv1822784994=39, srv1148938881=9, srv400101786=65, srv953797549=97, srv2117673955=53, srv644356081=76, srv1330584156=14, srv167025777=35, srv694328535=80, srv719951464=82, srv1293742068=12, srv526535763=71, srv1854628815=41, srv1454523552=22, srv158852500=31, srv765900903=87, srv735091052=84, srv1597181299=32, srv1329244192=13, srv257460250=58, srv690402850=79, srv1877917428=42, srv330940473=62, srv1087842375=5, srv458982904=67, srv46934440=68, srv1018241693=0, srv855456242=92, srv720801613=83, srv1432087377=21, srv737077315=85, srv2026000167=50, srv58490646=72, srv43763272=66, srv1336981486=15, srv2139179669=55, srv688346873=78, srv2122787615=54, srv1579717386=29, srv1758452109=38, srv233444953=57, srv2009908542=48, srv769299798=88, srv1952557611=45, srv1708254125=36, srv794582146=89, srv1109728096=7, srv2000186461=47, srv492323684=69, srv2037837768=51, srv323732350=61, srv1416015879=19, srv1576095731=28, srv1471479524=25, srv1188672681=10, srv596176104=74, srv1742813294=37, srv907918629=95, srv954490018=98, srv1374165157=17, srv810323536=90, srv1459026087=23, srv813943996=91, srv1108871989=6, srv707937541=81, srv890979605=93, srv739076226=86, srv1983792683=46, srv339580990=63, srv50305666=70, srv603666649=75, srv66252324=77, srv962223200=99, srv1421664875=20, srv1912567534=44, srv1666652875=34, srv1124423441=8, srv1581110249=30, srv1274615449=11, srv590454074=73, srv2058662995=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-13 22:53:08,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-13 22:53:08,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-13 22:53:08,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-13 22:53:08,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-13 22:53:08,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-13 22:53:08,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-13 22:53:08,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-13 22:53:08,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-13 22:53:08,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-13 22:53:08,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-13 22:53:08,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-13 22:53:08,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-13 22:53:08,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-13 22:53:08,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-13 22:53:08,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-13 22:53:08,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-13 22:53:08,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-13 22:53:08,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-13 22:53:08,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-13 22:53:08,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-13 22:53:08,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-13 22:53:08,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-13 22:53:08,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-13 22:53:08,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-13 22:53:08,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-13 22:53:08,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-13 22:53:08,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-13 22:53:08,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-13 22:53:08,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-13 22:53:08,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-13 22:53:08,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-13 22:53:08,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-13 22:53:08,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-13 22:53:08,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-13 22:53:08,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-13 22:53:08,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-13 22:53:08,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-13 22:53:08,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-13 22:53:08,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-13 22:53:08,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-13 22:53:08,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-13 22:53:08,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-13 22:53:08,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-13 22:53:08,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-13 22:53:08,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-13 22:53:08,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-13 22:53:08,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-13 22:53:08,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-13 22:53:08,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-13 22:53:08,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-13 22:53:08,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-13 22:53:08,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-13 22:53:08,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-13 22:53:08,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-13 22:53:08,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-13 22:53:08,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-13 22:53:08,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-13 22:53:08,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-13 22:53:08,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-13 22:53:08,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-13 22:53:08,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-13 22:53:08,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-13 22:53:08,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-13 22:53:08,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-13 22:53:08,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-13 22:53:08,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-13 22:53:08,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-13 22:53:08,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-13 22:53:08,730 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:08,730 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-13 22:53:08,731 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.1698208167017904E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-13 22:53:08,941 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 319 ms to try 5000 different iterations.  Found a solution that moves 31 regions; Going from a computed imbalance of 4.1698208167017904E-5 to a new imbalance of 6.961135693290261E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0034444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:08,942 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1188672681:83 , srv1330584156:83 , srv1581110249:83 , srv1845168347:83 , srv1877917428:83 , srv739076226:83 , srv810323536:83 , srv926398873:83 , srv1018241693:84 , srv1045009753:84 , srv107526439:84 , srv1076937052:84 , srv1084296280:84 , srv1087842375:84 , srv1108871989:84 , srv1109728096:84 , srv1124423441:84 , srv1148938881:84 , srv1274615449:84 , srv1293742068:84 , srv1329244192:84 , srv1336981486:84 , srv1365765018:84 , srv1380480853:84 , srv1416015879:84 , srv1421664875:84 , srv1432087377:84 , srv1454523552:84 , srv1459026087:84 , srv1460116671:84 , srv1471479524:84 , srv1473462474:84 , srv1510956296:84 , srv1576095731:84 , srv1579717386:84 , srv158852500:84 , srv1597181299:84 , srv1606939294:84 , srv1666652875:84 , srv167025777:84 , srv1708254125:84 , srv1758452109:84 , srv1822784994:84 , srv1854628815:84 , srv189982352:84 , srv1912567534:84 , srv1952557611:84 , srv1983792683:84 , srv2000186461:84 , srv2017899360:84 , srv2026000167:84 , srv2037837768:84 , srv2058662995:84 , srv2122787615:84 , srv2139179669:84 , srv216008860:84 , srv233444953:84 , srv257460250:84 , srv316401114:84 , srv323732350:84 , srv330940473:84 , srv339580990:84 , srv360163538:84 , srv400101786:84 , srv43763272:84 , srv458982904:84 , srv46934440:84 , srv492323684:84 , srv50305666:84 , srv526535763:84 , srv590454074:84 , srv596176104:84 , srv603666649:84 , srv644356081:84 , srv66252324:84 , srv688346873:84 , srv690402850:84 , srv694328535:84 , srv707937541:84 , srv719951464:84 , srv720801613:84 , srv735091052:84 , srv737077315:84 , srv765900903:84 , srv769299798:84 , srv794582146:84 , srv813943996:84 , srv855456242:84 , srv892303240:84 , srv907918629:84 , srv954490018:84 , srv1742813294:85 , srv2009908542:85 , srv2117673955:85 , srv263700835:85 , srv58490646:85 , srv890979605:85 , srv953797549:85 , srv962223200:85 , srv1374165157:684 }
2024-04-13 22:53:08,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-13 22:53:08,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1845168347=40, srv316401114=60, srv1045009753=1, srv1510956296=27, srv1365765018=16, srv216008860=56, srv263700835=59, srv189982352=43, srv1084296280=4, srv1380480853=18, srv2017899360=49, srv926398873=96, srv1460116671=24, srv1076937052=3, srv1606939294=33, srv892303240=94, srv107526439=2, srv1473462474=26, srv360163538=64, srv1822784994=39, srv1148938881=9, srv400101786=65, srv953797549=97, srv2117673955=53, srv644356081=76, srv1330584156=14, srv167025777=35, srv694328535=80, srv719951464=82, srv1293742068=12, srv526535763=71, srv1854628815=41, srv1454523552=22, srv158852500=31, srv765900903=87, srv735091052=84, srv1597181299=32, srv1329244192=13, srv257460250=58, srv690402850=79, srv1877917428=42, srv330940473=62, srv1087842375=5, srv458982904=67, srv46934440=68, srv1018241693=0, srv855456242=92, srv720801613=83, srv1432087377=21, srv737077315=85, srv2026000167=50, srv58490646=72, srv43763272=66, srv1336981486=15, srv2139179669=55, srv688346873=78, srv2122787615=54, srv1579717386=29, srv1758452109=38, srv233444953=57, srv2009908542=48, srv769299798=88, srv1952557611=45, srv1708254125=36, srv794582146=89, srv1109728096=7, srv2000186461=47, srv492323684=69, srv2037837768=51, srv323732350=61, srv1416015879=19, srv1576095731=28, srv1471479524=25, srv1188672681=10, srv596176104=74, srv1742813294=37, srv907918629=95, srv954490018=98, srv1374165157=17, srv810323536=90, srv1459026087=23, srv813943996=91, srv1108871989=6, srv707937541=81, srv890979605=93, srv739076226=86, srv1983792683=46, srv339580990=63, srv50305666=70, srv603666649=75, srv66252324=77, srv962223200=99, srv1421664875=20, srv1912567534=44, srv1666652875=34, srv1124423441=8, srv1581110249=30, srv1274615449=11, srv590454074=73, srv2058662995=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-13 22:53:08,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-13 22:53:08,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-13 22:53:08,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-13 22:53:08,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-13 22:53:08,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-13 22:53:08,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-13 22:53:08,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-13 22:53:08,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-13 22:53:08,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-13 22:53:08,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-13 22:53:08,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-13 22:53:08,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-13 22:53:09,065 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:09,066 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1188672681:83 , srv1330584156:83 , srv1581110249:83 , srv1845168347:83 , srv1877917428:83 , srv739076226:83 , srv810323536:83 , srv926398873:83 , srv1018241693:84 , srv1045009753:84 , srv107526439:84 , srv1076937052:84 , srv1084296280:84 , srv1087842375:84 , srv1108871989:84 , srv1109728096:84 , srv1124423441:84 , srv1148938881:84 , srv1274615449:84 , srv1293742068:84 , srv1329244192:84 , srv1336981486:84 , srv1365765018:84 , srv1380480853:84 , srv1416015879:84 , srv1421664875:84 , srv1432087377:84 , srv1454523552:84 , srv1459026087:84 , srv1460116671:84 , srv1471479524:84 , srv1473462474:84 , srv1510956296:84 , srv1576095731:84 , srv1579717386:84 , srv158852500:84 , srv1597181299:84 , srv1606939294:84 , srv1666652875:84 , srv167025777:84 , srv1708254125:84 , srv1758452109:84 , srv1822784994:84 , srv1854628815:84 , srv189982352:84 , srv1912567534:84 , srv1952557611:84 , srv1983792683:84 , srv2000186461:84 , srv2017899360:84 , srv2026000167:84 , srv2037837768:84 , srv2058662995:84 , srv2122787615:84 , srv2139179669:84 , srv216008860:84 , srv233444953:84 , srv257460250:84 , srv316401114:84 , srv323732350:84 , srv330940473:84 , srv339580990:84 , srv360163538:84 , srv400101786:84 , srv43763272:84 , srv458982904:84 , srv46934440:84 , srv492323684:84 , srv50305666:84 , srv526535763:84 , srv590454074:84 , srv596176104:84 , srv603666649:84 , srv644356081:84 , srv66252324:84 , srv688346873:84 , srv690402850:84 , srv694328535:84 , srv707937541:84 , srv719951464:84 , srv720801613:84 , srv735091052:84 , srv737077315:84 , srv765900903:84 , srv769299798:84 , srv794582146:84 , srv813943996:84 , srv855456242:84 , srv892303240:84 , srv907918629:84 , srv954490018:84 , srv1742813294:85 , srv2009908542:85 , srv2117673955:85 , srv263700835:85 , srv58490646:85 , srv890979605:85 , srv953797549:85 , srv962223200:85 , srv1374165157:684 }
2024-04-13 22:53:09,096 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=285 (was 285), ProcessCount=170 (was 170), AvailableMemoryMB=8081 (was 8824)
2024-04-13 22:53:09,105 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=285, ProcessCount=170, AvailableMemoryMB=8081
2024-04-13 22:53:09,106 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-13 22:53:09,106 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-04-13 22:53:09,122 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1055977363:84 , srv1230134192:84 , srv1334056911:84 , srv1344749361:84 , srv138087377:84 , srv139792742:84 , srv1522274953:84 , srv1528216579:84 , srv1602580101:84 , srv1649637188:84 , srv1733536793:84 , srv1792345765:84 , srv1943580696:84 , srv1944006711:84 , srv1948812504:84 , srv2142999636:84 , srv263322432:84 , srv295338039:84 , srv368067604:84 , srv39642800:84 , srv396471099:84 , srv501341163:84 , srv572146342:84 , srv610690460:84 , srv623231668:84 , srv768429740:84 , srv91016680:84 , srv990974820:84 , srv991517704:84 , srv877896931:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-04-13 22:53:09,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-13 22:53:09,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1943580696=12, srv1733536793=10, srv263322432=16, srv572146342=22, srv1792345765=11, srv1948812504=14, srv1528216579=7, srv1944006711=13, srv39642800=19, srv396471099=20, srv1649637188=9, srv991517704=29, srv768429740=25, srv1055977363=0, srv2142999636=15, srv139792742=5, srv138087377=4, srv295338039=17, srv91016680=27, srv610690460=23, srv1344749361=3, srv1602580101=8, srv501341163=21, srv1230134192=1, srv990974820=28, srv623231668=24, srv877896931=26, srv1522274953=6, srv1334056911=2, srv368067604=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-13 22:53:09,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-13 22:53:09,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-13 22:53:09,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-13 22:53:09,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-13 22:53:09,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-13 22:53:09,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-13 22:53:09,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-13 22:53:09,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-13 22:53:09,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-13 22:53:09,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-13 22:53:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-13 22:53:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-13 22:53:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-13 22:53:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-13 22:53:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-13 22:53:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-13 22:53:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-13 22:53:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-13 22:53:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-13 22:53:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-13 22:53:09,165 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1816371730809387, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:53:09,165 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912765376065656, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1816371730809387, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-04-13 22:55:09,144 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6136219 different iterations.  Found a solution that moves 1855 regions; Going from a computed imbalance of 0.9912765376065656 to a new imbalance of 4.329662539733032E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.687037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:55:09,152 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1055977363:90 , srv1230134192:90 , srv1334056911:90 , srv1344749361:90 , srv138087377:90 , srv139792742:90 , srv1522274953:90 , srv1528216579:90 , srv1602580101:90 , srv1649637188:90 , srv1733536793:90 , srv1792345765:90 , srv1943580696:90 , srv1944006711:90 , srv1948812504:90 , srv2142999636:90 , srv263322432:90 , srv295338039:90 , srv368067604:90 , srv39642800:90 , srv396471099:90 , srv501341163:90 , srv572146342:90 , srv610690460:90 , srv623231668:90 , srv768429740:90 , srv877896931:90 , srv91016680:90 , srv990974820:90 , srv991517704:90 }
2024-04-13 22:55:09,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-13 22:55:09,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1943580696=12, srv1733536793=10, srv263322432=16, srv572146342=22, srv1792345765=11, srv1948812504=14, srv1528216579=7, srv1944006711=13, srv39642800=19, srv396471099=20, srv1649637188=9, srv991517704=29, srv768429740=25, srv1055977363=0, srv2142999636=15, srv139792742=5, srv138087377=4, srv295338039=17, srv91016680=27, srv610690460=23, srv1344749361=3, srv1602580101=8, srv501341163=21, srv1230134192=1, srv990974820=28, srv623231668=24, srv877896931=26, srv1522274953=6, srv1334056911=2, srv368067604=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-13 22:55:09,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-13 22:55:09,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-13 22:55:09,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-13 22:55:09,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-13 22:55:09,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-13 22:55:09,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-13 22:55:09,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-13 22:55:09,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-13 22:55:09,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-13 22:55:09,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-13 22:55:09,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-13 22:55:09,169 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-13 22:55:09,170 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1055977363:90 , srv1230134192:90 , srv1334056911:90 , srv1344749361:90 , srv138087377:90 , srv139792742:90 , srv1522274953:90 , srv1528216579:90 , srv1602580101:90 , srv1649637188:90 , srv1733536793:90 , srv1792345765:90 , srv1943580696:90 , srv1944006711:90 , srv1948812504:90 , srv2142999636:90 , srv263322432:90 , srv295338039:90 , srv368067604:90 , srv39642800:90 , srv396471099:90 , srv501341163:90 , srv572146342:90 , srv610690460:90 , srv623231668:90 , srv768429740:90 , srv877896931:90 , srv91016680:90 , srv990974820:90 , srv991517704:90 }
2024-04-13 22:55:09,183 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 285), ProcessCount=167 (was 170), AvailableMemoryMB=7346 (was 8081)