2024-01-04 13:53:23,646 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-01-04 13:53:23,883 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-01-04 13:53:23,977 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-01-04 13:53:24,033 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-04 13:53:24,040 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-04 13:53:24,051 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=261, ProcessCount=168, AvailableMemoryMB=1995
2024-01-04 13:53:24,053 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-04 13:53:24,053 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-04 13:53:24,168 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1009016573:84 , srv1024060281:84 , srv1024827050:84 , srv102674858:84 , srv1046175288:84 , srv1126542822:84 , srv1197414947:84 , srv1221881811:84 , srv1228112953:84 , srv1237490631:84 , srv1241871567:84 , srv1258704950:84 , srv1261396733:84 , srv1273489280:84 , srv1289469201:84 , srv1303064117:84 , srv1346163229:84 , srv1369816119:84 , srv140565953:84 , srv1406181102:84 , srv1415923440:84 , srv1423394374:84 , srv1434596161:84 , srv1441000959:84 , srv146964044:84 , srv1492863708:84 , srv1499218091:84 , srv151595417:84 , srv1528234493:84 , srv1544468957:84 , srv1592601751:84 , srv1592615363:84 , srv1623156547:84 , srv1685632221:84 , srv1686301177:84 , srv1705606951:84 , srv1728306958:84 , srv1752894669:84 , srv1774775773:84 , srv1815437548:84 , srv1826608681:84 , srv184008313:84 , srv184945581:84 , srv1858068447:84 , srv1925183003:84 , srv1945405494:84 , srv1963868848:84 , srv1989876029:84 , srv2013394743:84 , srv202881060:84 , srv2077698239:84 , srv2079635659:84 , srv2092007346:84 , srv21829162:84 , srv279729918:84 , srv298854127:84 , srv306080121:84 , srv310439784:84 , srv31231275:84 , srv322842165:84 , srv329457001:84 , srv334306505:84 , srv343697008:84 , srv372457505:84 , srv380644578:84 , srv381411249:84 , srv402199684:84 , srv426489824:84 , srv440763972:84 , srv457165516:84 , srv487111870:84 , srv509847017:84 , srv509890073:84 , srv581726123:84 , srv61978038:84 , srv627476758:84 , srv636138392:84 , srv650478914:84 , srv661507989:84 , srv664613795:84 , srv66740905:84 , srv718784829:84 , srv724501782:84 , srv727862328:84 , srv773890934:84 , srv776634695:84 , srv790975216:84 , srv800257145:84 , srv811462736:84 , srv814524083:84 , srv847540236:84 , srv860381370:84 , srv865570010:84 , srv871218511:84 , srv891335837:84 , srv934022086:84 , srv938862772:84 , srv948191137:84 , srv98241641:84 , srv1406947787:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-01-04 13:53:24,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-04 13:53:24,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1406947787=20, srv322842165=60, srv1752894669=38, srv814524083=90, srv102674858=3, srv1826608681=41, srv1492863708=26, srv1197414947=6, srv1705606951=36, srv298854127=56, srv334306505=62, srv1858068447=44, srv1774775773=39, srv1945405494=46, srv61978038=75, srv871218511=94, srv343697008=63, srv372457505=64, srv2079635659=52, srv1592601751=31, srv279729918=55, srv509890073=73, srv776634695=86, srv1024827050=2, srv1544468957=30, srv938862772=97, srv1241871567=10, srv1925183003=45, srv184008313=42, srv664613795=80, srv66740905=81, srv1406181102=19, srv381411249=66, srv1989876029=48, srv811462736=89, srv1221881811=7, srv1369816119=17, srv1441000959=24, srv865570010=93, srv1528234493=29, srv306080121=57, srv636138392=77, srv426489824=68, srv380644578=65, srv581726123=74, srv1024060281=1, srv847540236=91, srv1592615363=32, srv1273489280=13, srv790975216=87, srv1963868848=47, srv31231275=59, srv2013394743=49, srv891335837=95, srv2092007346=53, srv202881060=50, srv440763972=69, srv457165516=70, srv1685632221=34, srv718784829=82, srv1415923440=21, srv724501782=83, srv509847017=72, srv1046175288=4, srv1009016573=0, srv146964044=25, srv800257145=88, srv184945581=43, srv1258704950=11, srv1499218091=27, srv773890934=85, srv934022086=96, srv310439784=58, srv329457001=61, srv1303064117=15, srv151595417=28, srv727862328=84, srv1423394374=22, srv2077698239=51, srv1815437548=40, srv1237490631=9, srv21829162=54, srv661507989=79, srv1261396733=12, srv487111870=71, srv1126542822=5, srv98241641=99, srv1686301177=35, srv402199684=67, srv1289469201=14, srv1623156547=33, srv1728306958=37, srv1346163229=16, srv1228112953=8, srv860381370=92, srv1434596161=23, srv650478914=78, srv627476758=76, srv140565953=18, srv948191137=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-04 13:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-04 13:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-04 13:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-04 13:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-04 13:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-04 13:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-04 13:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-04 13:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-04 13:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-04 13:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-04 13:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-04 13:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-04 13:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-04 13:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-04 13:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-04 13:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-04 13:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-04 13:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-04 13:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-04 13:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-04 13:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-04 13:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-04 13:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-04 13:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-04 13:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-04 13:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-04 13:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-04 13:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-04 13:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-04 13:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-04 13:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-04 13:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-04 13:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-04 13:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-04 13:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-04 13:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-04 13:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-04 13:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-04 13:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-04 13:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-04 13:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-04 13:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-04 13:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-04 13:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-04 13:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-04 13:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-04 13:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-04 13:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-04 13:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-04 13:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-04 13:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-04 13:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-04 13:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-04 13:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-04 13:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-04 13:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-04 13:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-04 13:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-04 13:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-04 13:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-04 13:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-04 13:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-04 13:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-04 13:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-04 13:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-04 13:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-04 13:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-04 13:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-04 13:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-04 13:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-04 13:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-04 13:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-04 13:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-04 13:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-04 13:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-04 13:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-04 13:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-04 13:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-04 13:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-04 13:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-04 13:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-04 13:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-04 13:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-04 13:53:24,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-04 13:53:24,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-04 13:53:24,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-04 13:53:24,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-04 13:53:24,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-04 13:53:24,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-04 13:53:24,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-04 13:53:24,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-04 13:53:24,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-04 13:53:24,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-04 13:53:24,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-04 13:53:24,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-04 13:53:24,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-04 13:53:24,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-04 13:53:24,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-04 13:53:24,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-04 13:53:24,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-04 13:53:24,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-04 13:53:24,409 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:24,409 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-04 13:53:24,409 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-04 13:53:24,724 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 452 ms to try 5000 different iterations.  Found a solution that moves 4692 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17547294117610346. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039776153808180614, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06048995612800629, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5213333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039776153808180614, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17075, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23608333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:24,738 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1406181102:81 , srv1728306958:81 , srv1945405494:81 , srv724501782:81 , srv790975216:81 , srv1289469201:82 , srv1544468957:82 , srv1685632221:82 , srv1752894669:82 , srv1826608681:82 , srv310439784:82 , srv31231275:82 , srv1126542822:83 , srv1492863708:83 , srv1774775773:83 , srv1963868848:83 , srv1989876029:83 , srv334306505:83 , srv381411249:83 , srv66740905:83 , srv98241641:83 , srv1623156547:84 , srv1925183003:84 , srv372457505:84 , srv664613795:84 , srv1009016573:85 , srv1237490631:85 , srv1273489280:85 , srv1415923440:85 , srv1434596161:85 , srv146964044:85 , srv1528234493:85 , srv1592601751:85 , srv426489824:85 , srv440763972:85 , srv487111870:85 , srv773890934:85 , srv776634695:85 , srv847540236:85 , srv938862772:85 , srv1346163229:86 , srv1369816119:86 , srv140565953:86 , srv1423394374:86 , srv151595417:86 , srv2013394743:86 , srv202881060:86 , srv329457001:86 , srv380644578:86 , srv636138392:86 , srv800257145:86 , srv891335837:86 , srv948191137:86 , srv1258704950:87 , srv1441000959:87 , srv1686301177:87 , srv1705606951:87 , srv184945581:87 , srv2092007346:87 , srv279729918:87 , srv298854127:87 , srv306080121:87 , srv343697008:87 , srv661507989:87 , srv727862328:87 , srv814524083:87 , srv871218511:87 , srv934022086:87 , srv1046175288:88 , srv1499218091:88 , srv1815437548:88 , srv1858068447:88 , srv457165516:88 , srv627476758:88 , srv650478914:88 , srv865570010:88 , srv1024060281:89 , srv1024827050:89 , srv102674858:90 , srv1221881811:90 , srv184008313:90 , srv2077698239:90 , srv402199684:90 , srv509890073:90 , srv581726123:90 , srv860381370:90 , srv1197414947:91 , srv1228112953:91 , srv1261396733:91 , srv1303064117:91 , srv2079635659:91 , srv61978038:91 , srv811462736:91 , srv1241871567:92 , srv21829162:92 , srv322842165:92 , srv1592615363:93 , srv509847017:93 , srv718784829:96 , srv1406947787:443 }
2024-01-04 13:53:24,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-04 13:53:24,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1406947787=20, srv322842165=60, srv1752894669=38, srv814524083=90, srv102674858=3, srv1826608681=41, srv1492863708=26, srv1197414947=6, srv1705606951=36, srv298854127=56, srv334306505=62, srv1858068447=44, srv1774775773=39, srv1945405494=46, srv61978038=75, srv871218511=94, srv343697008=63, srv372457505=64, srv2079635659=52, srv1592601751=31, srv279729918=55, srv509890073=73, srv776634695=86, srv1024827050=2, srv1544468957=30, srv938862772=97, srv1241871567=10, srv1925183003=45, srv184008313=42, srv664613795=80, srv66740905=81, srv1406181102=19, srv381411249=66, srv1989876029=48, srv811462736=89, srv1221881811=7, srv1369816119=17, srv1441000959=24, srv865570010=93, srv1528234493=29, srv306080121=57, srv636138392=77, srv426489824=68, srv380644578=65, srv581726123=74, srv1024060281=1, srv847540236=91, srv1592615363=32, srv1273489280=13, srv790975216=87, srv1963868848=47, srv31231275=59, srv2013394743=49, srv891335837=95, srv2092007346=53, srv202881060=50, srv440763972=69, srv457165516=70, srv1685632221=34, srv718784829=82, srv1415923440=21, srv724501782=83, srv509847017=72, srv1046175288=4, srv1009016573=0, srv146964044=25, srv800257145=88, srv184945581=43, srv1258704950=11, srv1499218091=27, srv773890934=85, srv934022086=96, srv310439784=58, srv329457001=61, srv1303064117=15, srv151595417=28, srv727862328=84, srv1423394374=22, srv2077698239=51, srv1815437548=40, srv1237490631=9, srv21829162=54, srv661507989=79, srv1261396733=12, srv487111870=71, srv1126542822=5, srv98241641=99, srv1686301177=35, srv402199684=67, srv1289469201=14, srv1623156547=33, srv1728306958=37, srv1346163229=16, srv1228112953=8, srv860381370=92, srv1434596161=23, srv650478914=78, srv627476758=76, srv140565953=18, srv948191137=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-04 13:53:24,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-04 13:53:24,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-04 13:53:24,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-04 13:53:24,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-04 13:53:24,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-04 13:53:24,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-04 13:53:24,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-04 13:53:24,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-04 13:53:24,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-04 13:53:24,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-04 13:53:24,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-04 13:53:24,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-04 13:53:24,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-04 13:53:24,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-04 13:53:24,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-04 13:53:24,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-04 13:53:24,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-04 13:53:24,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-04 13:53:24,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-04 13:53:24,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-04 13:53:24,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-04 13:53:24,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-04 13:53:24,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-04 13:53:24,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-04 13:53:24,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-04 13:53:24,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-04 13:53:24,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-04 13:53:24,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-04 13:53:24,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-04 13:53:24,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-04 13:53:24,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-04 13:53:24,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-04 13:53:24,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-04 13:53:24,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-04 13:53:24,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-04 13:53:24,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-04 13:53:24,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-04 13:53:24,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-04 13:53:24,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-04 13:53:24,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-04 13:53:24,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-04 13:53:24,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-04 13:53:24,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-04 13:53:24,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-04 13:53:24,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-04 13:53:24,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-04 13:53:24,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-04 13:53:24,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-04 13:53:24,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-04 13:53:24,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-04 13:53:24,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-04 13:53:24,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-04 13:53:24,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-04 13:53:24,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-04 13:53:24,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-04 13:53:24,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-04 13:53:24,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-04 13:53:24,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-04 13:53:24,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-04 13:53:24,897 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039776153808180614, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06048995612800629, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039776153808180614, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17075, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23608333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:24,897 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-04 13:53:24,897 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17544008708989903, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039776153808180614, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06048995612800629, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039776153808180614, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17075, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23608333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-04 13:53:25,069 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 273 ms to try 5000 different iterations.  Found a solution that moves 2617 regions; Going from a computed imbalance of 0.17544008708989903 to a new imbalance of 0.014496112166361413. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022609421341736182); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03710633248130625, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2907777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022609421341736182); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011333333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:25,076 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv718784829:75 , srv322842165:78 , srv1241871567:79 , srv1592615363:80 , srv509890073:80 , srv1024060281:81 , srv1221881811:81 , srv2077698239:81 , srv21829162:81 , srv509847017:81 , srv61978038:81 , srv102674858:82 , srv1228112953:82 , srv1261396733:82 , srv184008313:82 , srv2079635659:82 , srv581726123:82 , srv860381370:82 , srv1046175288:83 , srv1197414947:83 , srv1303064117:83 , srv1705606951:83 , srv1815437548:83 , srv306080121:83 , srv402199684:83 , srv650478914:83 , srv1024827050:84 , srv1346163229:84 , srv1499218091:84 , srv1686301177:84 , srv184945581:84 , srv1858068447:84 , srv457165516:84 , srv627476758:84 , srv727862328:84 , srv800257145:84 , srv140565953:85 , srv1423394374:85 , srv1441000959:85 , srv2092007346:85 , srv279729918:85 , srv298854127:85 , srv380644578:85 , srv636138392:85 , srv811462736:85 , srv847540236:85 , srv871218511:85 , srv934022086:85 , srv938862772:85 , srv1009016573:86 , srv1237490631:86 , srv1258704950:86 , srv1273489280:86 , srv151595417:86 , srv343697008:86 , srv440763972:86 , srv891335837:86 , srv1369816119:87 , srv1415923440:87 , srv1434596161:87 , srv146964044:87 , srv1623156547:87 , srv2013394743:87 , srv202881060:87 , srv372457505:87 , srv426489824:87 , srv661507989:87 , srv814524083:87 , srv865570010:87 , srv948191137:87 , srv1126542822:88 , srv1492863708:88 , srv1544468957:88 , srv1592601751:88 , srv1774775773:88 , srv381411249:88 , srv487111870:88 , srv773890934:88 , srv98241641:88 , srv1289469201:89 , srv1528234493:89 , srv1752894669:89 , srv1826608681:89 , srv1925183003:89 , srv1945405494:89 , srv1963868848:89 , srv329457001:89 , srv334306505:89 , srv664613795:89 , srv66740905:89 , srv724501782:89 , srv790975216:89 , srv1685632221:90 , srv310439784:90 , srv776634695:90 , srv1989876029:91 , srv1728306958:92 , srv1406181102:93 , srv31231275:93 , srv1406947787:532 }
2024-01-04 13:53:25,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-04 13:53:25,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1406947787=20, srv322842165=60, srv1752894669=38, srv814524083=90, srv102674858=3, srv1826608681=41, srv1492863708=26, srv1197414947=6, srv1705606951=36, srv298854127=56, srv334306505=62, srv1858068447=44, srv1774775773=39, srv1945405494=46, srv61978038=75, srv871218511=94, srv343697008=63, srv372457505=64, srv2079635659=52, srv1592601751=31, srv279729918=55, srv509890073=73, srv776634695=86, srv1024827050=2, srv1544468957=30, srv938862772=97, srv1241871567=10, srv1925183003=45, srv184008313=42, srv664613795=80, srv66740905=81, srv1406181102=19, srv381411249=66, srv1989876029=48, srv811462736=89, srv1221881811=7, srv1369816119=17, srv1441000959=24, srv865570010=93, srv1528234493=29, srv306080121=57, srv636138392=77, srv426489824=68, srv380644578=65, srv581726123=74, srv1024060281=1, srv847540236=91, srv1592615363=32, srv1273489280=13, srv790975216=87, srv1963868848=47, srv31231275=59, srv2013394743=49, srv891335837=95, srv2092007346=53, srv202881060=50, srv440763972=69, srv457165516=70, srv1685632221=34, srv718784829=82, srv1415923440=21, srv724501782=83, srv509847017=72, srv1046175288=4, srv1009016573=0, srv146964044=25, srv800257145=88, srv184945581=43, srv1258704950=11, srv1499218091=27, srv773890934=85, srv934022086=96, srv310439784=58, srv329457001=61, srv1303064117=15, srv151595417=28, srv727862328=84, srv1423394374=22, srv2077698239=51, srv1815437548=40, srv1237490631=9, srv21829162=54, srv661507989=79, srv1261396733=12, srv487111870=71, srv1126542822=5, srv98241641=99, srv1686301177=35, srv402199684=67, srv1289469201=14, srv1623156547=33, srv1728306958=37, srv1346163229=16, srv1228112953=8, srv860381370=92, srv1434596161=23, srv650478914=78, srv627476758=76, srv140565953=18, srv948191137=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-04 13:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-04 13:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-04 13:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-04 13:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-04 13:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-04 13:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-04 13:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-04 13:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-04 13:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-04 13:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-04 13:53:25,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-04 13:53:25,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-04 13:53:25,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-04 13:53:25,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-04 13:53:25,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-04 13:53:25,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-04 13:53:25,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-04 13:53:25,185 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022609421341736182); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03710633248130625, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022609421341736182); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011333333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:25,185 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-04 13:53:25,186 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014477787540701335, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022609421341736182); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03710633248130625, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022609421341736182); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011333333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-04 13:53:25,337 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 225 ms to try 5000 different iterations.  Found a solution that moves 984 regions; Going from a computed imbalance of 0.014477787540701335 to a new imbalance of 5.725570712002858E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006451832136876642); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008761716542303613); MoveCostFunction : (multiplier=7.0, imbalance=0.10933333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006451832136876642); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:25,339 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv31231275:81 , srv811462736:81 , srv329457001:82 , srv1197414947:83 , srv1303064117:83 , srv1406181102:83 , srv1528234493:83 , srv1989876029:83 , srv509847017:83 , srv776634695:83 , srv865570010:83 , srv1261396733:84 , srv1592615363:84 , srv1925183003:84 , srv2013394743:84 , srv202881060:84 , srv343697008:84 , srv627476758:84 , srv661507989:84 , srv66740905:84 , srv814524083:84 , srv1024827050:85 , srv1221881811:85 , srv1228112953:85 , srv1258704950:85 , srv1273489280:85 , srv1369816119:85 , srv1441000959:85 , srv146964044:85 , srv1499218091:85 , srv151595417:85 , srv1592601751:85 , srv1685632221:85 , srv1728306958:85 , srv1858068447:85 , srv1963868848:85 , srv21829162:85 , srv279729918:85 , srv298854127:85 , srv334306505:85 , srv402199684:85 , srv426489824:85 , srv581726123:85 , srv61978038:85 , srv773890934:85 , srv948191137:85 , srv98241641:85 , srv1009016573:86 , srv102674858:86 , srv1046175288:86 , srv1237490631:86 , srv1415923440:86 , srv1423394374:86 , srv1434596161:86 , srv1492863708:86 , srv1686301177:86 , srv1752894669:86 , srv1774775773:86 , srv1815437548:86 , srv1826608681:86 , srv184008313:86 , srv184945581:86 , srv2077698239:86 , srv2079635659:86 , srv2092007346:86 , srv372457505:86 , srv380644578:86 , srv457165516:86 , srv487111870:86 , srv650478914:86 , srv718784829:86 , srv871218511:86 , srv891335837:86 , srv934022086:86 , srv1024060281:87 , srv1126542822:87 , srv1241871567:87 , srv1289469201:87 , srv1346163229:87 , srv140565953:87 , srv1623156547:87 , srv1705606951:87 , srv306080121:87 , srv322842165:87 , srv381411249:87 , srv636138392:87 , srv664613795:87 , srv724501782:87 , srv727862328:87 , srv790975216:87 , srv800257145:87 , srv847540236:87 , srv860381370:87 , srv938862772:87 , srv1544468957:88 , srv1945405494:88 , srv310439784:88 , srv440763972:88 , srv509890073:88 , srv1406947787:540 }
2024-01-04 13:53:25,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-04 13:53:25,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1406947787=20, srv322842165=60, srv1752894669=38, srv814524083=90, srv102674858=3, srv1826608681=41, srv1492863708=26, srv1197414947=6, srv1705606951=36, srv298854127=56, srv334306505=62, srv1858068447=44, srv1774775773=39, srv1945405494=46, srv61978038=75, srv871218511=94, srv343697008=63, srv372457505=64, srv2079635659=52, srv1592601751=31, srv279729918=55, srv509890073=73, srv776634695=86, srv1024827050=2, srv1544468957=30, srv938862772=97, srv1241871567=10, srv1925183003=45, srv184008313=42, srv664613795=80, srv66740905=81, srv1406181102=19, srv381411249=66, srv1989876029=48, srv811462736=89, srv1221881811=7, srv1369816119=17, srv1441000959=24, srv865570010=93, srv1528234493=29, srv306080121=57, srv636138392=77, srv426489824=68, srv380644578=65, srv581726123=74, srv1024060281=1, srv847540236=91, srv1592615363=32, srv1273489280=13, srv790975216=87, srv1963868848=47, srv31231275=59, srv2013394743=49, srv891335837=95, srv2092007346=53, srv202881060=50, srv440763972=69, srv457165516=70, srv1685632221=34, srv718784829=82, srv1415923440=21, srv724501782=83, srv509847017=72, srv1046175288=4, srv1009016573=0, srv146964044=25, srv800257145=88, srv184945581=43, srv1258704950=11, srv1499218091=27, srv773890934=85, srv934022086=96, srv310439784=58, srv329457001=61, srv1303064117=15, srv151595417=28, srv727862328=84, srv1423394374=22, srv2077698239=51, srv1815437548=40, srv1237490631=9, srv21829162=54, srv661507989=79, srv1261396733=12, srv487111870=71, srv1126542822=5, srv98241641=99, srv1686301177=35, srv402199684=67, srv1289469201=14, srv1623156547=33, srv1728306958=37, srv1346163229=16, srv1228112953=8, srv860381370=92, srv1434596161=23, srv650478914=78, srv627476758=76, srv140565953=18, srv948191137=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-04 13:53:25,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-04 13:53:25,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-04 13:53:25,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-04 13:53:25,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-04 13:53:25,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-04 13:53:25,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-04 13:53:25,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-04 13:53:25,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-04 13:53:25,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-04 13:53:25,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-04 13:53:25,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-04 13:53:25,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-04 13:53:25,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-04 13:53:25,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-04 13:53:25,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-04 13:53:25,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-04 13:53:25,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-04 13:53:25,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-04 13:53:25,437 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006451832136876642); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008761716542303613); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006451832136876642); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:25,438 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-04 13:53:25,438 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.656669559348993E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006451832136876642); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008761716542303613); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006451832136876642); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-04 13:53:25,653 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 288 ms to try 5000 different iterations.  Found a solution that moves 240 regions; Going from a computed imbalance of 5.656669559348993E-4 to a new imbalance of 2.8015188665258175E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.02666666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:25,653 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv310439784:82 , srv2079635659:83 , srv440763972:83 , srv487111870:83 , srv664613795:83 , srv860381370:83 , srv871218511:83 , srv1009016573:84 , srv1024060281:84 , srv1024827050:84 , srv102674858:84 , srv1046175288:84 , srv1126542822:84 , srv1228112953:84 , srv1241871567:84 , srv1258704950:84 , srv1289469201:84 , srv1369816119:84 , srv140565953:84 , srv1415923440:84 , srv1434596161:84 , srv1441000959:84 , srv151595417:84 , srv1544468957:84 , srv1623156547:84 , srv1774775773:84 , srv184008313:84 , srv1945405494:84 , srv202881060:84 , srv2092007346:84 , srv21829162:84 , srv381411249:84 , srv457165516:84 , srv509890073:84 , srv636138392:84 , srv661507989:84 , srv727862328:84 , srv773890934:84 , srv776634695:84 , srv811462736:84 , srv814524083:84 , srv865570010:84 , srv934022086:84 , srv948191137:84 , srv1197414947:85 , srv1237490631:85 , srv1261396733:85 , srv1303064117:85 , srv1346163229:85 , srv1406181102:85 , srv1423394374:85 , srv146964044:85 , srv1492863708:85 , srv1499218091:85 , srv1528234493:85 , srv1592601751:85 , srv1592615363:85 , srv1685632221:85 , srv1686301177:85 , srv1705606951:85 , srv1728306958:85 , srv1752894669:85 , srv1815437548:85 , srv1826608681:85 , srv184945581:85 , srv1858068447:85 , srv1925183003:85 , srv1963868848:85 , srv1989876029:85 , srv2013394743:85 , srv298854127:85 , srv329457001:85 , srv334306505:85 , srv343697008:85 , srv372457505:85 , srv380644578:85 , srv402199684:85 , srv426489824:85 , srv509847017:85 , srv581726123:85 , srv61978038:85 , srv650478914:85 , srv718784829:85 , srv724501782:85 , srv790975216:85 , srv800257145:85 , srv847540236:85 , srv891335837:85 , srv938862772:85 , srv1221881811:86 , srv1273489280:86 , srv2077698239:86 , srv279729918:86 , srv306080121:86 , srv31231275:86 , srv322842165:86 , srv627476758:86 , srv66740905:86 , srv98241641:86 , srv1406947787:627 }
2024-01-04 13:53:25,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-04 13:53:25,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1406947787=20, srv322842165=60, srv1752894669=38, srv814524083=90, srv102674858=3, srv1826608681=41, srv1492863708=26, srv1197414947=6, srv1705606951=36, srv298854127=56, srv334306505=62, srv1858068447=44, srv1774775773=39, srv1945405494=46, srv61978038=75, srv871218511=94, srv343697008=63, srv372457505=64, srv2079635659=52, srv1592601751=31, srv279729918=55, srv509890073=73, srv776634695=86, srv1024827050=2, srv1544468957=30, srv938862772=97, srv1241871567=10, srv1925183003=45, srv184008313=42, srv664613795=80, srv66740905=81, srv1406181102=19, srv381411249=66, srv1989876029=48, srv811462736=89, srv1221881811=7, srv1369816119=17, srv1441000959=24, srv865570010=93, srv1528234493=29, srv306080121=57, srv636138392=77, srv426489824=68, srv380644578=65, srv581726123=74, srv1024060281=1, srv847540236=91, srv1592615363=32, srv1273489280=13, srv790975216=87, srv1963868848=47, srv31231275=59, srv2013394743=49, srv891335837=95, srv2092007346=53, srv202881060=50, srv440763972=69, srv457165516=70, srv1685632221=34, srv718784829=82, srv1415923440=21, srv724501782=83, srv509847017=72, srv1046175288=4, srv1009016573=0, srv146964044=25, srv800257145=88, srv184945581=43, srv1258704950=11, srv1499218091=27, srv773890934=85, srv934022086=96, srv310439784=58, srv329457001=61, srv1303064117=15, srv151595417=28, srv727862328=84, srv1423394374=22, srv2077698239=51, srv1815437548=40, srv1237490631=9, srv21829162=54, srv661507989=79, srv1261396733=12, srv487111870=71, srv1126542822=5, srv98241641=99, srv1686301177=35, srv402199684=67, srv1289469201=14, srv1623156547=33, srv1728306958=37, srv1346163229=16, srv1228112953=8, srv860381370=92, srv1434596161=23, srv650478914=78, srv627476758=76, srv140565953=18, srv948191137=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-04 13:53:25,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-04 13:53:25,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-04 13:53:25,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-04 13:53:25,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-04 13:53:25,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-04 13:53:25,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-04 13:53:25,767 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:25,768 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-04 13:53:25,768 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.6334672746871227E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-04 13:53:25,911 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 227 ms to try 5000 different iterations.  Found a solution that moves 27 regions; Going from a computed imbalance of 2.6334672746871227E-5 to a new imbalance of 7.262297337147979E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.003); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:25,911 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1592601751:83 , srv1728306958:83 , srv2077698239:83 , srv279729918:83 , srv306080121:83 , srv322842165:83 , srv402199684:83 , srv664613795:83 , srv891335837:83 , srv1024827050:84 , srv102674858:84 , srv1126542822:84 , srv1197414947:84 , srv1221881811:84 , srv1228112953:84 , srv1237490631:84 , srv1241871567:84 , srv1258704950:84 , srv1261396733:84 , srv1273489280:84 , srv1289469201:84 , srv1303064117:84 , srv1346163229:84 , srv1369816119:84 , srv140565953:84 , srv1406181102:84 , srv1415923440:84 , srv1423394374:84 , srv1434596161:84 , srv146964044:84 , srv1492863708:84 , srv1499218091:84 , srv1528234493:84 , srv1544468957:84 , srv1592615363:84 , srv1623156547:84 , srv1685632221:84 , srv1686301177:84 , srv1705606951:84 , srv1752894669:84 , srv1815437548:84 , srv1826608681:84 , srv184008313:84 , srv184945581:84 , srv1858068447:84 , srv1925183003:84 , srv1945405494:84 , srv1963868848:84 , srv1989876029:84 , srv2013394743:84 , srv2079635659:84 , srv2092007346:84 , srv21829162:84 , srv298854127:84 , srv310439784:84 , srv31231275:84 , srv329457001:84 , srv334306505:84 , srv343697008:84 , srv372457505:84 , srv380644578:84 , srv381411249:84 , srv426489824:84 , srv440763972:84 , srv457165516:84 , srv487111870:84 , srv509847017:84 , srv509890073:84 , srv581726123:84 , srv61978038:84 , srv627476758:84 , srv636138392:84 , srv650478914:84 , srv661507989:84 , srv66740905:84 , srv718784829:84 , srv724501782:84 , srv727862328:84 , srv773890934:84 , srv776634695:84 , srv790975216:84 , srv800257145:84 , srv814524083:84 , srv847540236:84 , srv860381370:84 , srv865570010:84 , srv934022086:84 , srv938862772:84 , srv948191137:84 , srv98241641:84 , srv1009016573:85 , srv1024060281:85 , srv1046175288:85 , srv1441000959:85 , srv151595417:85 , srv1774775773:85 , srv202881060:85 , srv811462736:85 , srv871218511:85 , srv1406947787:684 }
2024-01-04 13:53:25,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-04 13:53:25,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1406947787=20, srv322842165=60, srv1752894669=38, srv814524083=90, srv102674858=3, srv1826608681=41, srv1492863708=26, srv1197414947=6, srv1705606951=36, srv298854127=56, srv334306505=62, srv1858068447=44, srv1774775773=39, srv1945405494=46, srv61978038=75, srv871218511=94, srv343697008=63, srv372457505=64, srv2079635659=52, srv1592601751=31, srv279729918=55, srv509890073=73, srv776634695=86, srv1024827050=2, srv1544468957=30, srv938862772=97, srv1241871567=10, srv1925183003=45, srv184008313=42, srv664613795=80, srv66740905=81, srv1406181102=19, srv381411249=66, srv1989876029=48, srv811462736=89, srv1221881811=7, srv1369816119=17, srv1441000959=24, srv865570010=93, srv1528234493=29, srv306080121=57, srv636138392=77, srv426489824=68, srv380644578=65, srv581726123=74, srv1024060281=1, srv847540236=91, srv1592615363=32, srv1273489280=13, srv790975216=87, srv1963868848=47, srv31231275=59, srv2013394743=49, srv891335837=95, srv2092007346=53, srv202881060=50, srv440763972=69, srv457165516=70, srv1685632221=34, srv718784829=82, srv1415923440=21, srv724501782=83, srv509847017=72, srv1046175288=4, srv1009016573=0, srv146964044=25, srv800257145=88, srv184945581=43, srv1258704950=11, srv1499218091=27, srv773890934=85, srv934022086=96, srv310439784=58, srv329457001=61, srv1303064117=15, srv151595417=28, srv727862328=84, srv1423394374=22, srv2077698239=51, srv1815437548=40, srv1237490631=9, srv21829162=54, srv661507989=79, srv1261396733=12, srv487111870=71, srv1126542822=5, srv98241641=99, srv1686301177=35, srv402199684=67, srv1289469201=14, srv1623156547=33, srv1728306958=37, srv1346163229=16, srv1228112953=8, srv860381370=92, srv1434596161=23, srv650478914=78, srv627476758=76, srv140565953=18, srv948191137=98} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-04 13:53:25,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-04 13:53:25,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-04 13:53:25,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-04 13:53:25,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-04 13:53:25,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-04 13:53:25,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-04 13:53:26,016 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:26,016 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1592601751:83 , srv1728306958:83 , srv2077698239:83 , srv279729918:83 , srv306080121:83 , srv322842165:83 , srv402199684:83 , srv664613795:83 , srv891335837:83 , srv1024827050:84 , srv102674858:84 , srv1126542822:84 , srv1197414947:84 , srv1221881811:84 , srv1228112953:84 , srv1237490631:84 , srv1241871567:84 , srv1258704950:84 , srv1261396733:84 , srv1273489280:84 , srv1289469201:84 , srv1303064117:84 , srv1346163229:84 , srv1369816119:84 , srv140565953:84 , srv1406181102:84 , srv1415923440:84 , srv1423394374:84 , srv1434596161:84 , srv146964044:84 , srv1492863708:84 , srv1499218091:84 , srv1528234493:84 , srv1544468957:84 , srv1592615363:84 , srv1623156547:84 , srv1685632221:84 , srv1686301177:84 , srv1705606951:84 , srv1752894669:84 , srv1815437548:84 , srv1826608681:84 , srv184008313:84 , srv184945581:84 , srv1858068447:84 , srv1925183003:84 , srv1945405494:84 , srv1963868848:84 , srv1989876029:84 , srv2013394743:84 , srv2079635659:84 , srv2092007346:84 , srv21829162:84 , srv298854127:84 , srv310439784:84 , srv31231275:84 , srv329457001:84 , srv334306505:84 , srv343697008:84 , srv372457505:84 , srv380644578:84 , srv381411249:84 , srv426489824:84 , srv440763972:84 , srv457165516:84 , srv487111870:84 , srv509847017:84 , srv509890073:84 , srv581726123:84 , srv61978038:84 , srv627476758:84 , srv636138392:84 , srv650478914:84 , srv661507989:84 , srv66740905:84 , srv718784829:84 , srv724501782:84 , srv727862328:84 , srv773890934:84 , srv776634695:84 , srv790975216:84 , srv800257145:84 , srv814524083:84 , srv847540236:84 , srv860381370:84 , srv865570010:84 , srv934022086:84 , srv938862772:84 , srv948191137:84 , srv98241641:84 , srv1009016573:85 , srv1024060281:85 , srv1046175288:85 , srv1441000959:85 , srv151595417:85 , srv1774775773:85 , srv202881060:85 , srv811462736:85 , srv871218511:85 , srv1406947787:684 }
2024-01-04 13:53:26,035 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=256 (was 261), ProcessCount=168 (was 168), AvailableMemoryMB=1524 (was 1995)
2024-01-04 13:53:26,041 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=256, ProcessCount=168, AvailableMemoryMB=1523
2024-01-04 13:53:26,042 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-04 13:53:26,042 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-01-04 13:53:26,049 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1057213959:84 , srv1297834109:84 , srv1328616764:84 , srv1331377886:84 , srv1345494092:84 , srv140336916:84 , srv1451156756:84 , srv1506523678:84 , srv1625219193:84 , srv1631078510:84 , srv1660439271:84 , srv1849509443:84 , srv2043556750:84 , srv2050841965:84 , srv209110237:84 , srv2124181024:84 , srv33612174:84 , srv409783839:84 , srv588864556:84 , srv685740595:84 , srv700034414:84 , srv716075389:84 , srv758432564:84 , srv761492642:84 , srv762920291:84 , srv790134064:84 , srv824415994:84 , srv856667186:84 , srv953645711:84 , srv1354266402:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-01-04 13:53:26,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-04 13:53:26,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1849509443=12, srv856667186=28, srv1328616764=2, srv1345494092=4, srv1631078510=10, srv762920291=25, srv33612174=17, srv716075389=22, srv1331377886=3, srv758432564=23, srv953645711=29, srv1451156756=7, srv209110237=15, srv790134064=26, srv588864556=19, srv761492642=24, srv2124181024=16, srv700034414=21, srv824415994=27, srv1354266402=5, srv1660439271=11, srv2050841965=14, srv1297834109=1, srv2043556750=13, srv1506523678=8, srv140336916=6, srv685740595=20, srv1625219193=9, srv1057213959=0, srv409783839=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-04 13:53:26,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-04 13:53:26,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-04 13:53:26,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-04 13:53:26,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-04 13:53:26,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-04 13:53:26,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-04 13:53:26,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-04 13:53:26,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-04 13:53:26,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-04 13:53:26,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-04 13:53:26,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-04 13:53:26,071 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1984056424272453, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:53:26,071 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912818212965026, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1984056424272453, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-01-04 13:55:26,064 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9547725 different iterations.  Found a solution that moves 1846 regions; Going from a computed imbalance of 0.9912818212965026 to a new imbalance of 4.308656090753194E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6837037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:55:26,070 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1057213959:90 , srv1297834109:90 , srv1328616764:90 , srv1331377886:90 , srv1345494092:90 , srv1354266402:90 , srv140336916:90 , srv1451156756:90 , srv1506523678:90 , srv1625219193:90 , srv1631078510:90 , srv1660439271:90 , srv1849509443:90 , srv2043556750:90 , srv2050841965:90 , srv209110237:90 , srv2124181024:90 , srv33612174:90 , srv409783839:90 , srv588864556:90 , srv685740595:90 , srv700034414:90 , srv716075389:90 , srv758432564:90 , srv761492642:90 , srv762920291:90 , srv790134064:90 , srv824415994:90 , srv856667186:90 , srv953645711:90 }
2024-01-04 13:55:26,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-04 13:55:26,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1849509443=12, srv856667186=28, srv1328616764=2, srv1345494092=4, srv1631078510=10, srv762920291=25, srv33612174=17, srv716075389=22, srv1331377886=3, srv758432564=23, srv953645711=29, srv1451156756=7, srv209110237=15, srv790134064=26, srv588864556=19, srv761492642=24, srv2124181024=16, srv700034414=21, srv824415994=27, srv1354266402=5, srv1660439271=11, srv2050841965=14, srv1297834109=1, srv2043556750=13, srv1506523678=8, srv140336916=6, srv685740595=20, srv1625219193=9, srv1057213959=0, srv409783839=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-04 13:55:26,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-04 13:55:26,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-04 13:55:26,087 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-04 13:55:26,087 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1057213959:90 , srv1297834109:90 , srv1328616764:90 , srv1331377886:90 , srv1345494092:90 , srv1354266402:90 , srv140336916:90 , srv1451156756:90 , srv1506523678:90 , srv1625219193:90 , srv1631078510:90 , srv1660439271:90 , srv1849509443:90 , srv2043556750:90 , srv2050841965:90 , srv209110237:90 , srv2124181024:90 , srv33612174:90 , srv409783839:90 , srv588864556:90 , srv685740595:90 , srv700034414:90 , srv716075389:90 , srv758432564:90 , srv761492642:90 , srv762920291:90 , srv790134064:90 , srv824415994:90 , srv856667186:90 , srv953645711:90 }
2024-01-04 13:55:26,098 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=157 (was 256), ProcessCount=166 (was 168), AvailableMemoryMB=1167 (was 1523)