2023-10-19 02:53:29,517 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-19 02:53:29,854 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-19 02:53:29,958 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-19 02:53:30,035 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-19 02:53:30,043 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-19 02:53:30,058 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=296, ProcessCount=170, AvailableMemoryMB=11268
2023-10-19 02:53:30,060 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-19 02:53:30,060 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-19 02:53:30,182 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1027433062:84 , srv1034430730:84 , srv1083794854:84 , srv1085566053:84 , srv1111717619:84 , srv1154928491:84 , srv115761537:84 , srv1203066126:84 , srv1220251959:84 , srv1240525583:84 , srv1246518384:84 , srv128107099:84 , srv1283884662:84 , srv1303336719:84 , srv134165809:84 , srv1345770096:84 , srv1412777993:84 , srv1427701576:84 , srv147539059:84 , srv1489266384:84 , srv1500283615:84 , srv1527485569:84 , srv1540635910:84 , srv1558644308:84 , srv1564335620:84 , srv1568300980:84 , srv1599215342:84 , srv1620404923:84 , srv1624196028:84 , srv162534218:84 , srv1654211965:84 , srv1655707735:84 , srv1660309228:84 , srv1690253014:84 , srv1696984310:84 , srv173713653:84 , srv1758661496:84 , srv176570388:84 , srv180848335:84 , srv1823113262:84 , srv182945747:84 , srv1829592514:84 , srv1829896336:84 , srv1845910107:84 , srv1851573415:84 , srv1900048637:84 , srv1917735607:84 , srv1919918449:84 , srv1950983729:84 , srv1957421396:84 , srv1961440946:84 , srv1992671279:84 , srv1995436811:84 , srv2024341602:84 , srv2028281560:84 , srv2030525787:84 , srv2038269477:84 , srv2075385457:84 , srv2077749827:84 , srv2087337706:84 , srv2089611481:84 , srv211802299:84 , srv2156747:84 , srv233711047:84 , srv252289834:84 , srv256581142:84 , srv303425085:84 , srv317775789:84 , srv373304754:84 , srv394587581:84 , srv400730316:84 , srv402956024:84 , srv458003747:84 , srv462660802:84 , srv469084259:84 , srv486521034:84 , srv491922115:84 , srv516040581:84 , srv527989086:84 , srv549257500:84 , srv553070199:84 , srv565445273:84 , srv61033944:84 , srv628301873:84 , srv637262794:84 , srv67283252:84 , srv674186877:84 , srv681110525:84 , srv681465088:84 , srv696705637:84 , srv70275852:84 , srv714942706:84 , srv738566156:84 , srv862782561:84 , srv878199834:84 , srv88439820:84 , srv913659519:84 , srv941789029:84 , srv9472082:84 , srv462966102:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-19 02:53:30,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 02:53:30,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv61033944=83, srv1568300980=25, srv9472082=99, srv549257500=80, srv252289834=64, srv67283252=86, srv233711047=63, srv628301873=84, srv373304754=68, srv256581142=65, srv1500283615=20, srv1489266384=19, srv527989086=79, srv1034430730=1, srv180848335=38, srv1845910107=43, srv681110525=88, srv1527485569=21, srv1696984310=34, srv1085566053=3, srv2028281560=54, srv878199834=95, srv1919918449=47, srv1303336719=13, srv1654211965=30, srv462966102=74, srv637262794=85, srv1540635910=22, srv1961440946=50, srv1851573415=44, srv941789029=98, srv2087337706=59, srv2089611481=60, srv1564335620=24, srv1758661496=36, srv1655707735=31, srv1620404923=27, srv458003747=72, srv115761537=6, srv1345770096=15, srv1427701576=17, srv1154928491=5, srv173713653=35, srv738566156=93, srv1083794854=2, srv2077749827=58, srv317775789=67, srv128107099=11, srv1246518384=10, srv1412777993=16, srv469084259=75, srv516040581=78, srv1283884662=12, srv553070199=81, srv176570388=37, srv862782561=94, srv1240525583=9, srv913659519=97, srv696705637=90, srv134165809=14, srv70275852=91, srv2030525787=55, srv462660802=73, srv1829896336=42, srv2038269477=56, srv1690253014=33, srv1900048637=45, srv1917735607=46, srv1992671279=51, srv2024341602=53, srv681465088=89, srv1624196028=28, srv1203066126=7, srv211802299=61, srv1660309228=32, srv1995436811=52, srv1111717619=4, srv491922115=77, srv1599215342=26, srv147539059=18, srv565445273=82, srv88439820=96, srv394587581=69, srv2156747=62, srv674186877=87, srv182945747=40, srv714942706=92, srv400730316=70, srv1027433062=0, srv402956024=71, srv1957421396=49, srv1950983729=48, srv1220251959=8, srv1558644308=23, srv1823113262=39, srv2075385457=57, srv486521034=76, srv303425085=66, srv1829592514=41, srv162534218=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 02:53:30,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 02:53:30,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 02:53:30,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 02:53:30,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 02:53:30,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 02:53:30,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 02:53:30,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 02:53:30,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 02:53:30,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 02:53:30,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 02:53:30,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 02:53:30,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 02:53:30,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 02:53:30,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 02:53:30,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 02:53:30,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 02:53:30,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 02:53:30,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 02:53:30,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 02:53:30,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 02:53:30,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 02:53:30,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 02:53:30,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 02:53:30,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 02:53:30,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 02:53:30,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 02:53:30,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 02:53:30,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 02:53:30,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 02:53:30,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 02:53:30,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 02:53:30,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 02:53:30,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 02:53:30,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 02:53:30,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 02:53:30,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 02:53:30,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 02:53:30,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 02:53:30,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 02:53:30,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 02:53:30,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 02:53:30,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 02:53:30,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 02:53:30,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 02:53:30,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 02:53:30,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 02:53:30,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 02:53:30,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 02:53:30,450 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:30,450 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-19 02:53:30,450 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-19 02:53:30,789 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 489 ms to try 5000 different iterations.  Found a solution that moves 4688 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17880954481177616. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04069357539206035, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05908771466803703, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5208888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04069357539206035, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17383333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24233333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:30,805 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1085566053:82 , srv1489266384:82 , srv1654211965:82 , srv70275852:82 , srv1540635910:83 , srv180848335:83 , srv182945747:83 , srv486521034:83 , srv491922115:83 , srv1303336719:84 , srv1599215342:84 , srv1624196028:84 , srv1823113262:84 , srv1919918449:84 , srv1961440946:84 , srv2024341602:84 , srv2038269477:84 , srv252289834:84 , srv303425085:84 , srv458003747:84 , srv738566156:84 , srv878199834:84 , srv913659519:84 , srv1027433062:85 , srv162534218:85 , srv176570388:85 , srv211802299:85 , srv317775789:85 , srv373304754:85 , srv394587581:85 , srv549257500:85 , srv637262794:85 , srv681110525:85 , srv941789029:85 , srv1034430730:86 , srv1111717619:86 , srv1154928491:86 , srv128107099:86 , srv1345770096:86 , srv1829592514:86 , srv1829896336:86 , srv1845910107:86 , srv1992671279:86 , srv2028281560:86 , srv2089611481:86 , srv469084259:86 , srv674186877:86 , srv1083794854:87 , srv115761537:87 , srv1220251959:87 , srv1240525583:87 , srv1246518384:87 , srv1527485569:87 , srv1558644308:87 , srv1655707735:87 , srv1660309228:87 , srv173713653:87 , srv1900048637:87 , srv2075385457:87 , srv2156747:87 , srv527989086:87 , srv553070199:87 , srv565445273:87 , srv61033944:87 , srv628301873:87 , srv681465088:87 , srv696705637:87 , srv88439820:87 , srv1203066126:88 , srv1427701576:88 , srv147539059:88 , srv1564335620:88 , srv1568300980:88 , srv1957421396:88 , srv1995436811:88 , srv2077749827:88 , srv402956024:88 , srv462660802:88 , srv516040581:88 , srv67283252:88 , srv714942706:88 , srv862782561:88 , srv1283884662:89 , srv134165809:89 , srv1500283615:89 , srv1620404923:89 , srv1690253014:89 , srv1696984310:89 , srv1758661496:89 , srv1851573415:89 , srv1917735607:89 , srv1950983729:89 , srv2030525787:89 , srv256581142:89 , srv400730316:89 , srv9472082:89 , srv2087337706:90 , srv233711047:90 , srv1412777993:91 , srv462966102:452 }
2023-10-19 02:53:30,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 02:53:30,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv61033944=83, srv1568300980=25, srv9472082=99, srv549257500=80, srv252289834=64, srv67283252=86, srv233711047=63, srv628301873=84, srv373304754=68, srv256581142=65, srv1500283615=20, srv1489266384=19, srv527989086=79, srv1034430730=1, srv180848335=38, srv1845910107=43, srv681110525=88, srv1527485569=21, srv1696984310=34, srv1085566053=3, srv2028281560=54, srv878199834=95, srv1919918449=47, srv1303336719=13, srv1654211965=30, srv462966102=74, srv637262794=85, srv1540635910=22, srv1961440946=50, srv1851573415=44, srv941789029=98, srv2087337706=59, srv2089611481=60, srv1564335620=24, srv1758661496=36, srv1655707735=31, srv1620404923=27, srv458003747=72, srv115761537=6, srv1345770096=15, srv1427701576=17, srv1154928491=5, srv173713653=35, srv738566156=93, srv1083794854=2, srv2077749827=58, srv317775789=67, srv128107099=11, srv1246518384=10, srv1412777993=16, srv469084259=75, srv516040581=78, srv1283884662=12, srv553070199=81, srv176570388=37, srv862782561=94, srv1240525583=9, srv913659519=97, srv696705637=90, srv134165809=14, srv70275852=91, srv2030525787=55, srv462660802=73, srv1829896336=42, srv2038269477=56, srv1690253014=33, srv1900048637=45, srv1917735607=46, srv1992671279=51, srv2024341602=53, srv681465088=89, srv1624196028=28, srv1203066126=7, srv211802299=61, srv1660309228=32, srv1995436811=52, srv1111717619=4, srv491922115=77, srv1599215342=26, srv147539059=18, srv565445273=82, srv88439820=96, srv394587581=69, srv2156747=62, srv674186877=87, srv182945747=40, srv714942706=92, srv400730316=70, srv1027433062=0, srv402956024=71, srv1957421396=49, srv1950983729=48, srv1220251959=8, srv1558644308=23, srv1823113262=39, srv2075385457=57, srv486521034=76, srv303425085=66, srv1829592514=41, srv162534218=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 02:53:30,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 02:53:30,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 02:53:30,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 02:53:30,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 02:53:30,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 02:53:30,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 02:53:30,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 02:53:30,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 02:53:30,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 02:53:30,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 02:53:30,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 02:53:30,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 02:53:30,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 02:53:30,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 02:53:30,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 02:53:30,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 02:53:30,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 02:53:30,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 02:53:30,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 02:53:30,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 02:53:30,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 02:53:30,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 02:53:30,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 02:53:30,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 02:53:30,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 02:53:30,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 02:53:30,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 02:53:30,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 02:53:30,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 02:53:30,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 02:53:30,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 02:53:30,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 02:53:30,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 02:53:30,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 02:53:30,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 02:53:30,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 02:53:30,958 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04069357539206035, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05908771466803703, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04069357539206035, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17383333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24233333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:30,958 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-19 02:53:30,958 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17877671873417034, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04069357539206035, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05908771466803703, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04069357539206035, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17383333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24233333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-19 02:53:31,218 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 364 ms to try 5000 different iterations.  Found a solution that moves 2623 regions; Going from a computed imbalance of 0.17877671873417034 to a new imbalance of 0.015631287981690178. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024520339277507464); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03723616503524479, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29144444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024520339277507464); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04691666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:31,227 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1758661496:80 , srv1950983729:80 , srv1412777993:81 , srv1917735607:81 , srv1283884662:82 , srv134165809:82 , srv1427701576:82 , srv1500283615:82 , srv1696984310:82 , srv1851573415:82 , srv2030525787:82 , srv2087337706:82 , srv256581142:82 , srv400730316:82 , srv462660802:82 , srv862782561:82 , srv1220251959:83 , srv1564335620:83 , srv1995436811:83 , srv2077749827:83 , srv2156747:83 , srv402956024:83 , srv553070199:83 , srv67283252:83 , srv1154928491:84 , srv115761537:84 , srv1203066126:84 , srv1568300980:84 , srv1655707735:84 , srv1690253014:84 , srv173713653:84 , srv1900048637:84 , srv233711047:84 , srv469084259:84 , srv516040581:84 , srv527989086:84 , srv681465088:84 , srv696705637:84 , srv88439820:84 , srv1240525583:85 , srv128107099:85 , srv1345770096:85 , srv1558644308:85 , srv1620404923:85 , srv1829592514:85 , srv2028281560:85 , srv2075385457:85 , srv2089611481:85 , srv211802299:85 , srv394587581:85 , srv637262794:85 , srv9472082:85 , srv1083794854:86 , srv1527485569:86 , srv1599215342:86 , srv1845910107:86 , srv1919918449:86 , srv1957421396:86 , srv628301873:86 , srv738566156:86 , srv1027433062:87 , srv1034430730:87 , srv1111717619:87 , srv1246518384:87 , srv162534218:87 , srv1660309228:87 , srv1829896336:87 , srv1992671279:87 , srv2024341602:87 , srv303425085:87 , srv486521034:87 , srv549257500:87 , srv565445273:87 , srv681110525:87 , srv714942706:87 , srv941789029:87 , srv1303336719:88 , srv147539059:88 , srv1540635910:88 , srv176570388:88 , srv1823113262:88 , srv1961440946:88 , srv2038269477:88 , srv252289834:88 , srv317775789:88 , srv373304754:88 , srv458003747:88 , srv61033944:88 , srv1624196028:89 , srv180848335:89 , srv182945747:89 , srv70275852:89 , srv491922115:90 , srv674186877:90 , srv878199834:90 , srv913659519:90 , srv1085566053:91 , srv1489266384:91 , srv1654211965:91 , srv462966102:540 }
2023-10-19 02:53:31,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 02:53:31,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv61033944=83, srv1568300980=25, srv9472082=99, srv549257500=80, srv252289834=64, srv67283252=86, srv233711047=63, srv628301873=84, srv373304754=68, srv256581142=65, srv1500283615=20, srv1489266384=19, srv527989086=79, srv1034430730=1, srv180848335=38, srv1845910107=43, srv681110525=88, srv1527485569=21, srv1696984310=34, srv1085566053=3, srv2028281560=54, srv878199834=95, srv1919918449=47, srv1303336719=13, srv1654211965=30, srv462966102=74, srv637262794=85, srv1540635910=22, srv1961440946=50, srv1851573415=44, srv941789029=98, srv2087337706=59, srv2089611481=60, srv1564335620=24, srv1758661496=36, srv1655707735=31, srv1620404923=27, srv458003747=72, srv115761537=6, srv1345770096=15, srv1427701576=17, srv1154928491=5, srv173713653=35, srv738566156=93, srv1083794854=2, srv2077749827=58, srv317775789=67, srv128107099=11, srv1246518384=10, srv1412777993=16, srv469084259=75, srv516040581=78, srv1283884662=12, srv553070199=81, srv176570388=37, srv862782561=94, srv1240525583=9, srv913659519=97, srv696705637=90, srv134165809=14, srv70275852=91, srv2030525787=55, srv462660802=73, srv1829896336=42, srv2038269477=56, srv1690253014=33, srv1900048637=45, srv1917735607=46, srv1992671279=51, srv2024341602=53, srv681465088=89, srv1624196028=28, srv1203066126=7, srv211802299=61, srv1660309228=32, srv1995436811=52, srv1111717619=4, srv491922115=77, srv1599215342=26, srv147539059=18, srv565445273=82, srv88439820=96, srv394587581=69, srv2156747=62, srv674186877=87, srv182945747=40, srv714942706=92, srv400730316=70, srv1027433062=0, srv402956024=71, srv1957421396=49, srv1950983729=48, srv1220251959=8, srv1558644308=23, srv1823113262=39, srv2075385457=57, srv486521034=76, srv303425085=66, srv1829592514=41, srv162534218=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 02:53:31,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 02:53:31,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 02:53:31,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 02:53:31,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 02:53:31,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 02:53:31,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 02:53:31,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 02:53:31,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 02:53:31,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 02:53:31,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 02:53:31,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 02:53:31,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 02:53:31,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 02:53:31,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 02:53:31,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 02:53:31,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 02:53:31,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 02:53:31,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 02:53:31,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 02:53:31,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 02:53:31,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 02:53:31,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 02:53:31,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 02:53:31,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 02:53:31,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 02:53:31,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 02:53:31,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 02:53:31,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 02:53:31,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 02:53:31,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 02:53:31,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 02:53:31,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 02:53:31,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 02:53:31,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 02:53:31,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 02:53:31,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 02:53:31,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 02:53:31,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 02:53:31,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 02:53:31,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 02:53:31,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 02:53:31,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 02:53:31,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 02:53:31,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 02:53:31,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 02:53:31,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 02:53:31,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 02:53:31,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 02:53:31,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 02:53:31,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 02:53:31,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 02:53:31,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 02:53:31,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 02:53:31,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 02:53:31,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 02:53:31,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 02:53:31,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 02:53:31,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 02:53:31,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 02:53:31,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 02:53:31,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 02:53:31,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 02:53:31,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 02:53:31,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 02:53:31,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 02:53:31,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 02:53:31,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 02:53:31,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 02:53:31,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 02:53:31,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 02:53:31,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 02:53:31,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 02:53:31,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 02:53:31,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 02:53:31,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 02:53:31,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 02:53:31,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 02:53:31,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 02:53:31,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 02:53:31,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 02:53:31,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 02:53:31,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 02:53:31,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 02:53:31,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 02:53:31,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 02:53:31,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 02:53:31,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 02:53:31,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 02:53:31,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 02:53:31,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 02:53:31,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 02:53:31,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 02:53:31,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 02:53:31,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 02:53:31,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 02:53:31,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 02:53:31,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 02:53:31,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 02:53:31,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 02:53:31,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 02:53:31,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 02:53:31,388 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024520339277507464); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03723616503524479, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024520339277507464); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04691666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:31,388 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-19 02:53:31,388 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01561292134313214, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024520339277507464); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03723616503524479, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024520339277507464); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04691666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-19 02:53:31,654 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 372 ms to try 5000 different iterations.  Found a solution that moves 941 regions; Going from a computed imbalance of 0.01561292134313214 to a new imbalance of 0.001193258133603718. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009466370534352253); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.015160939829732299); MoveCostFunction : (multiplier=7.0, imbalance=0.10455555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.009466370534352253); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:31,657 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv147539059:80 , srv61033944:81 , srv674186877:81 , srv1620404923:82 , srv565445273:82 , srv714942706:82 , srv1246518384:83 , srv1489266384:83 , srv878199834:83 , srv1034430730:84 , srv1083794854:84 , srv1085566053:84 , srv1111717619:84 , srv1527485569:84 , srv1624196028:84 , srv1654211965:84 , srv1660309228:84 , srv180848335:84 , srv1829896336:84 , srv1957421396:84 , srv1961440946:84 , srv1992671279:84 , srv2087337706:84 , srv317775789:84 , srv373304754:84 , srv549257500:84 , srv913659519:84 , srv9472082:84 , srv1027433062:85 , srv1240525583:85 , srv1412777993:85 , srv1500283615:85 , srv1558644308:85 , srv162534218:85 , srv1690253014:85 , srv173713653:85 , srv176570388:85 , srv1823113262:85 , srv182945747:85 , srv1845910107:85 , srv2038269477:85 , srv233711047:85 , srv252289834:85 , srv516040581:85 , srv628301873:85 , srv67283252:85 , srv681110525:85 , srv70275852:85 , srv941789029:85 , srv1154928491:86 , srv1203066126:86 , srv1220251959:86 , srv128107099:86 , srv1283884662:86 , srv1303336719:86 , srv134165809:86 , srv1345770096:86 , srv1540635910:86 , srv1564335620:86 , srv1568300980:86 , srv1655707735:86 , srv1696984310:86 , srv1851573415:86 , srv1919918449:86 , srv1995436811:86 , srv2024341602:86 , srv2028281560:86 , srv2030525787:86 , srv2075385457:86 , srv2077749827:86 , srv256581142:86 , srv303425085:86 , srv400730316:86 , srv402956024:86 , srv458003747:86 , srv491922115:86 , srv738566156:86 , srv862782561:86 , srv115761537:87 , srv1427701576:87 , srv1599215342:87 , srv1829592514:87 , srv1900048637:87 , srv1917735607:87 , srv2089611481:87 , srv469084259:87 , srv527989086:87 , srv553070199:87 , srv696705637:87 , srv88439820:87 , srv1758661496:88 , srv1950983729:88 , srv211802299:88 , srv2156747:88 , srv394587581:88 , srv637262794:88 , srv681465088:88 , srv462660802:89 , srv486521034:89 , srv462966102:550 }
2023-10-19 02:53:31,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 02:53:31,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv61033944=83, srv1568300980=25, srv9472082=99, srv549257500=80, srv252289834=64, srv67283252=86, srv233711047=63, srv628301873=84, srv373304754=68, srv256581142=65, srv1500283615=20, srv1489266384=19, srv527989086=79, srv1034430730=1, srv180848335=38, srv1845910107=43, srv681110525=88, srv1527485569=21, srv1696984310=34, srv1085566053=3, srv2028281560=54, srv878199834=95, srv1919918449=47, srv1303336719=13, srv1654211965=30, srv462966102=74, srv637262794=85, srv1540635910=22, srv1961440946=50, srv1851573415=44, srv941789029=98, srv2087337706=59, srv2089611481=60, srv1564335620=24, srv1758661496=36, srv1655707735=31, srv1620404923=27, srv458003747=72, srv115761537=6, srv1345770096=15, srv1427701576=17, srv1154928491=5, srv173713653=35, srv738566156=93, srv1083794854=2, srv2077749827=58, srv317775789=67, srv128107099=11, srv1246518384=10, srv1412777993=16, srv469084259=75, srv516040581=78, srv1283884662=12, srv553070199=81, srv176570388=37, srv862782561=94, srv1240525583=9, srv913659519=97, srv696705637=90, srv134165809=14, srv70275852=91, srv2030525787=55, srv462660802=73, srv1829896336=42, srv2038269477=56, srv1690253014=33, srv1900048637=45, srv1917735607=46, srv1992671279=51, srv2024341602=53, srv681465088=89, srv1624196028=28, srv1203066126=7, srv211802299=61, srv1660309228=32, srv1995436811=52, srv1111717619=4, srv491922115=77, srv1599215342=26, srv147539059=18, srv565445273=82, srv88439820=96, srv394587581=69, srv2156747=62, srv674186877=87, srv182945747=40, srv714942706=92, srv400730316=70, srv1027433062=0, srv402956024=71, srv1957421396=49, srv1950983729=48, srv1220251959=8, srv1558644308=23, srv1823113262=39, srv2075385457=57, srv486521034=76, srv303425085=66, srv1829592514=41, srv162534218=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 02:53:31,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 02:53:31,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 02:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 02:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 02:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 02:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 02:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 02:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 02:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 02:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 02:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 02:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 02:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 02:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 02:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 02:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 02:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 02:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 02:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 02:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 02:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 02:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 02:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 02:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 02:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 02:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 02:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 02:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 02:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 02:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 02:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 02:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 02:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 02:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 02:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 02:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 02:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 02:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 02:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 02:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 02:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 02:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 02:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 02:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 02:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 02:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 02:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 02:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 02:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 02:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 02:53:31,813 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009466370534352253); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.015160939829732299); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.009466370534352253); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:31,814 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-19 02:53:31,814 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.001186669110773709, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009466370534352253); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.015160939829732299); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.009466370534352253); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-19 02:53:32,178 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 477 ms to try 5000 different iterations.  Found a solution that moves 294 regions; Going from a computed imbalance of 0.001186669110773709 to a new imbalance of 3.5838749626429105E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.03266666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:32,180 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv233711047:83 , srv486521034:83 , srv491922115:83 , srv696705637:83 , srv115761537:84 , srv1203066126:84 , srv1303336719:84 , srv134165809:84 , srv1540635910:84 , srv1568300980:84 , srv1660309228:84 , srv1690253014:84 , srv1696984310:84 , srv176570388:84 , srv1829592514:84 , srv1829896336:84 , srv2075385457:84 , srv211802299:84 , srv2156747:84 , srv303425085:84 , srv394587581:84 , srv400730316:84 , srv458003747:84 , srv462660802:84 , srv527989086:84 , srv637262794:84 , srv681465088:84 , srv88439820:84 , srv913659519:84 , srv941789029:84 , srv9472082:84 , srv1027433062:85 , srv1083794854:85 , srv1085566053:85 , srv1111717619:85 , srv1220251959:85 , srv1240525583:85 , srv1246518384:85 , srv128107099:85 , srv1283884662:85 , srv1412777993:85 , srv1489266384:85 , srv1500283615:85 , srv1527485569:85 , srv1558644308:85 , srv1564335620:85 , srv1599215342:85 , srv1624196028:85 , srv162534218:85 , srv1654211965:85 , srv1655707735:85 , srv1758661496:85 , srv182945747:85 , srv1845910107:85 , srv1851573415:85 , srv1900048637:85 , srv1917735607:85 , srv1919918449:85 , srv1950983729:85 , srv1957421396:85 , srv1992671279:85 , srv1995436811:85 , srv2028281560:85 , srv2030525787:85 , srv2077749827:85 , srv2089611481:85 , srv252289834:85 , srv256581142:85 , srv317775789:85 , srv373304754:85 , srv402956024:85 , srv516040581:85 , srv553070199:85 , srv628301873:85 , srv674186877:85 , srv681110525:85 , srv714942706:85 , srv862782561:85 , srv878199834:85 , srv1034430730:86 , srv1154928491:86 , srv1345770096:86 , srv1427701576:86 , srv147539059:86 , srv1620404923:86 , srv173713653:86 , srv180848335:86 , srv1823113262:86 , srv1961440946:86 , srv2024341602:86 , srv2038269477:86 , srv2087337706:86 , srv469084259:86 , srv549257500:86 , srv565445273:86 , srv61033944:86 , srv67283252:86 , srv70275852:86 , srv738566156:86 , srv462966102:600 }
2023-10-19 02:53:32,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 02:53:32,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv61033944=83, srv1568300980=25, srv9472082=99, srv549257500=80, srv252289834=64, srv67283252=86, srv233711047=63, srv628301873=84, srv373304754=68, srv256581142=65, srv1500283615=20, srv1489266384=19, srv527989086=79, srv1034430730=1, srv180848335=38, srv1845910107=43, srv681110525=88, srv1527485569=21, srv1696984310=34, srv1085566053=3, srv2028281560=54, srv878199834=95, srv1919918449=47, srv1303336719=13, srv1654211965=30, srv462966102=74, srv637262794=85, srv1540635910=22, srv1961440946=50, srv1851573415=44, srv941789029=98, srv2087337706=59, srv2089611481=60, srv1564335620=24, srv1758661496=36, srv1655707735=31, srv1620404923=27, srv458003747=72, srv115761537=6, srv1345770096=15, srv1427701576=17, srv1154928491=5, srv173713653=35, srv738566156=93, srv1083794854=2, srv2077749827=58, srv317775789=67, srv128107099=11, srv1246518384=10, srv1412777993=16, srv469084259=75, srv516040581=78, srv1283884662=12, srv553070199=81, srv176570388=37, srv862782561=94, srv1240525583=9, srv913659519=97, srv696705637=90, srv134165809=14, srv70275852=91, srv2030525787=55, srv462660802=73, srv1829896336=42, srv2038269477=56, srv1690253014=33, srv1900048637=45, srv1917735607=46, srv1992671279=51, srv2024341602=53, srv681465088=89, srv1624196028=28, srv1203066126=7, srv211802299=61, srv1660309228=32, srv1995436811=52, srv1111717619=4, srv491922115=77, srv1599215342=26, srv147539059=18, srv565445273=82, srv88439820=96, srv394587581=69, srv2156747=62, srv674186877=87, srv182945747=40, srv714942706=92, srv400730316=70, srv1027433062=0, srv402956024=71, srv1957421396=49, srv1950983729=48, srv1220251959=8, srv1558644308=23, srv1823113262=39, srv2075385457=57, srv486521034=76, srv303425085=66, srv1829592514=41, srv162534218=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 02:53:32,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 02:53:32,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 02:53:32,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 02:53:32,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 02:53:32,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 02:53:32,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 02:53:32,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 02:53:32,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 02:53:32,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 02:53:32,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 02:53:32,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 02:53:32,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 02:53:32,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 02:53:32,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 02:53:32,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 02:53:32,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 02:53:32,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 02:53:32,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 02:53:32,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 02:53:32,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 02:53:32,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 02:53:32,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 02:53:32,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 02:53:32,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 02:53:32,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 02:53:32,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 02:53:32,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 02:53:32,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 02:53:32,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 02:53:32,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 02:53:32,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 02:53:32,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 02:53:32,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 02:53:32,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 02:53:32,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 02:53:32,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 02:53:32,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 02:53:32,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 02:53:32,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 02:53:32,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 02:53:32,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 02:53:32,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 02:53:32,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 02:53:32,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 02:53:32,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 02:53:32,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 02:53:32,335 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:32,335 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-19 02:53:32,335 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.37801176264051E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-19 02:53:32,580 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 358 ms to try 5000 different iterations.  Found a solution that moves 33 regions; Going from a computed imbalance of 3.37801176264051E-5 to a new imbalance of 7.618828439090763E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0036666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:32,581 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1034430730:83 , srv1345770096:83 , srv1427701576:83 , srv1823113262:83 , srv1900048637:83 , srv1957421396:83 , srv2024341602:83 , srv2038269477:83 , srv2089611481:83 , srv462660802:83 , srv469084259:83 , srv628301873:83 , srv681465088:83 , srv1027433062:84 , srv1083794854:84 , srv1085566053:84 , srv1111717619:84 , srv1154928491:84 , srv115761537:84 , srv1203066126:84 , srv1240525583:84 , srv1246518384:84 , srv128107099:84 , srv1283884662:84 , srv1303336719:84 , srv1412777993:84 , srv147539059:84 , srv1527485569:84 , srv1558644308:84 , srv1564335620:84 , srv1568300980:84 , srv1599215342:84 , srv1620404923:84 , srv1624196028:84 , srv162534218:84 , srv1654211965:84 , srv1655707735:84 , srv1660309228:84 , srv1690253014:84 , srv173713653:84 , srv1758661496:84 , srv176570388:84 , srv180848335:84 , srv182945747:84 , srv1829592514:84 , srv1845910107:84 , srv1851573415:84 , srv1917735607:84 , srv1950983729:84 , srv1961440946:84 , srv1992671279:84 , srv1995436811:84 , srv2028281560:84 , srv2030525787:84 , srv2075385457:84 , srv2077749827:84 , srv2087337706:84 , srv211802299:84 , srv2156747:84 , srv233711047:84 , srv252289834:84 , srv256581142:84 , srv317775789:84 , srv373304754:84 , srv394587581:84 , srv402956024:84 , srv458003747:84 , srv486521034:84 , srv491922115:84 , srv516040581:84 , srv527989086:84 , srv549257500:84 , srv553070199:84 , srv565445273:84 , srv61033944:84 , srv637262794:84 , srv67283252:84 , srv674186877:84 , srv681110525:84 , srv70275852:84 , srv714942706:84 , srv738566156:84 , srv878199834:84 , srv88439820:84 , srv913659519:84 , srv9472082:84 , srv1220251959:85 , srv134165809:85 , srv1489266384:85 , srv1500283615:85 , srv1540635910:85 , srv1696984310:85 , srv1829896336:85 , srv1919918449:85 , srv303425085:85 , srv400730316:85 , srv696705637:85 , srv862782561:85 , srv941789029:85 , srv462966102:684 }
2023-10-19 02:53:32,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 02:53:32,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv61033944=83, srv1568300980=25, srv9472082=99, srv549257500=80, srv252289834=64, srv67283252=86, srv233711047=63, srv628301873=84, srv373304754=68, srv256581142=65, srv1500283615=20, srv1489266384=19, srv527989086=79, srv1034430730=1, srv180848335=38, srv1845910107=43, srv681110525=88, srv1527485569=21, srv1696984310=34, srv1085566053=3, srv2028281560=54, srv878199834=95, srv1919918449=47, srv1303336719=13, srv1654211965=30, srv462966102=74, srv637262794=85, srv1540635910=22, srv1961440946=50, srv1851573415=44, srv941789029=98, srv2087337706=59, srv2089611481=60, srv1564335620=24, srv1758661496=36, srv1655707735=31, srv1620404923=27, srv458003747=72, srv115761537=6, srv1345770096=15, srv1427701576=17, srv1154928491=5, srv173713653=35, srv738566156=93, srv1083794854=2, srv2077749827=58, srv317775789=67, srv128107099=11, srv1246518384=10, srv1412777993=16, srv469084259=75, srv516040581=78, srv1283884662=12, srv553070199=81, srv176570388=37, srv862782561=94, srv1240525583=9, srv913659519=97, srv696705637=90, srv134165809=14, srv70275852=91, srv2030525787=55, srv462660802=73, srv1829896336=42, srv2038269477=56, srv1690253014=33, srv1900048637=45, srv1917735607=46, srv1992671279=51, srv2024341602=53, srv681465088=89, srv1624196028=28, srv1203066126=7, srv211802299=61, srv1660309228=32, srv1995436811=52, srv1111717619=4, srv491922115=77, srv1599215342=26, srv147539059=18, srv565445273=82, srv88439820=96, srv394587581=69, srv2156747=62, srv674186877=87, srv182945747=40, srv714942706=92, srv400730316=70, srv1027433062=0, srv402956024=71, srv1957421396=49, srv1950983729=48, srv1220251959=8, srv1558644308=23, srv1823113262=39, srv2075385457=57, srv486521034=76, srv303425085=66, srv1829592514=41, srv162534218=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 02:53:32,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 02:53:32,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 02:53:32,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 02:53:32,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 02:53:32,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 02:53:32,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 02:53:32,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 02:53:32,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 02:53:32,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 02:53:32,728 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:32,728 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1034430730:83 , srv1345770096:83 , srv1427701576:83 , srv1823113262:83 , srv1900048637:83 , srv1957421396:83 , srv2024341602:83 , srv2038269477:83 , srv2089611481:83 , srv462660802:83 , srv469084259:83 , srv628301873:83 , srv681465088:83 , srv1027433062:84 , srv1083794854:84 , srv1085566053:84 , srv1111717619:84 , srv1154928491:84 , srv115761537:84 , srv1203066126:84 , srv1240525583:84 , srv1246518384:84 , srv128107099:84 , srv1283884662:84 , srv1303336719:84 , srv1412777993:84 , srv147539059:84 , srv1527485569:84 , srv1558644308:84 , srv1564335620:84 , srv1568300980:84 , srv1599215342:84 , srv1620404923:84 , srv1624196028:84 , srv162534218:84 , srv1654211965:84 , srv1655707735:84 , srv1660309228:84 , srv1690253014:84 , srv173713653:84 , srv1758661496:84 , srv176570388:84 , srv180848335:84 , srv182945747:84 , srv1829592514:84 , srv1845910107:84 , srv1851573415:84 , srv1917735607:84 , srv1950983729:84 , srv1961440946:84 , srv1992671279:84 , srv1995436811:84 , srv2028281560:84 , srv2030525787:84 , srv2075385457:84 , srv2077749827:84 , srv2087337706:84 , srv211802299:84 , srv2156747:84 , srv233711047:84 , srv252289834:84 , srv256581142:84 , srv317775789:84 , srv373304754:84 , srv394587581:84 , srv402956024:84 , srv458003747:84 , srv486521034:84 , srv491922115:84 , srv516040581:84 , srv527989086:84 , srv549257500:84 , srv553070199:84 , srv565445273:84 , srv61033944:84 , srv637262794:84 , srv67283252:84 , srv674186877:84 , srv681110525:84 , srv70275852:84 , srv714942706:84 , srv738566156:84 , srv878199834:84 , srv88439820:84 , srv913659519:84 , srv9472082:84 , srv1220251959:85 , srv134165809:85 , srv1489266384:85 , srv1500283615:85 , srv1540635910:85 , srv1696984310:85 , srv1829896336:85 , srv1919918449:85 , srv303425085:85 , srv400730316:85 , srv696705637:85 , srv862782561:85 , srv941789029:85 , srv462966102:684 }
2023-10-19 02:53:32,763 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=296 (was 296), ProcessCount=170 (was 170), AvailableMemoryMB=10756 (was 11268)
2023-10-19 02:53:32,772 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=296, ProcessCount=170, AvailableMemoryMB=10756
2023-10-19 02:53:32,772 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-19 02:53:32,773 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-19 02:53:32,784 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1124643970:84 , srv1413757438:84 , srv1457603986:84 , srv1478622124:84 , srv153110341:84 , srv1647498715:84 , srv1702738790:84 , srv1707186301:84 , srv1778016183:84 , srv1818026503:84 , srv1853525550:84 , srv1907266087:84 , srv2076676949:84 , srv212984447:84 , srv2138762391:84 , srv229397711:84 , srv256497820:84 , srv27022306:84 , srv308633051:84 , srv341009239:84 , srv357622914:84 , srv529355719:84 , srv557668335:84 , srv583702376:84 , srv751613510:84 , srv761066589:84 , srv880342113:84 , srv890926227:84 , srv997090222:84 , srv1800874102:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-19 02:53:32,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 02:53:32,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv212984447=14, srv880342113=27, srv1702738790=6, srv1647498715=5, srv341009239=20, srv357622914=21, srv1457603986=2, srv1707186301=7, srv229397711=16, srv308633051=19, srv2076676949=13, srv761066589=26, srv153110341=4, srv1907266087=12, srv1778016183=8, srv529355719=22, srv2138762391=15, srv256497820=17, srv1800874102=9, srv1818026503=10, srv1478622124=3, srv557668335=23, srv751613510=25, srv27022306=18, srv1413757438=1, srv890926227=28, srv583702376=24, srv1124643970=0, srv1853525550=11, srv997090222=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 02:53:32,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 02:53:32,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 02:53:32,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 02:53:32,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 02:53:32,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 02:53:32,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 02:53:32,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 02:53:32,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 02:53:32,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 02:53:32,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 02:53:32,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 02:53:32,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 02:53:32,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 02:53:32,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 02:53:32,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 02:53:32,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 02:53:32,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 02:53:32,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 02:53:32,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-19 02:53:32,822 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.116209603405787, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:53:32,822 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912559215930019, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.116209603405787, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-19 02:55:32,806 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6171956 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912559215930019 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:55:32,813 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1124643970:90 , srv1413757438:90 , srv1457603986:90 , srv1478622124:90 , srv153110341:90 , srv1647498715:90 , srv1702738790:90 , srv1707186301:90 , srv1778016183:90 , srv1800874102:90 , srv1818026503:90 , srv1853525550:90 , srv1907266087:90 , srv2076676949:90 , srv212984447:90 , srv2138762391:90 , srv229397711:90 , srv256497820:90 , srv27022306:90 , srv308633051:90 , srv341009239:90 , srv357622914:90 , srv529355719:90 , srv557668335:90 , srv583702376:90 , srv751613510:90 , srv761066589:90 , srv880342113:90 , srv890926227:90 , srv997090222:90 }
2023-10-19 02:55:32,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 02:55:32,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv212984447=14, srv880342113=27, srv1702738790=6, srv1647498715=5, srv341009239=20, srv357622914=21, srv1457603986=2, srv1707186301=7, srv229397711=16, srv308633051=19, srv2076676949=13, srv761066589=26, srv153110341=4, srv1907266087=12, srv1778016183=8, srv529355719=22, srv2138762391=15, srv256497820=17, srv1800874102=9, srv1818026503=10, srv1478622124=3, srv557668335=23, srv751613510=25, srv27022306=18, srv1413757438=1, srv890926227=28, srv583702376=24, srv1124643970=0, srv1853525550=11, srv997090222=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 02:55:32,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 02:55:32,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 02:55:32,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 02:55:32,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 02:55:32,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 02:55:32,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 02:55:32,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 02:55:32,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 02:55:32,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 02:55:32,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 02:55:32,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-19 02:55:32,833 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 02:55:32,833 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1124643970:90 , srv1413757438:90 , srv1457603986:90 , srv1478622124:90 , srv153110341:90 , srv1647498715:90 , srv1702738790:90 , srv1707186301:90 , srv1778016183:90 , srv1800874102:90 , srv1818026503:90 , srv1853525550:90 , srv1907266087:90 , srv2076676949:90 , srv212984447:90 , srv2138762391:90 , srv229397711:90 , srv256497820:90 , srv27022306:90 , srv308633051:90 , srv341009239:90 , srv357622914:90 , srv529355719:90 , srv557668335:90 , srv583702376:90 , srv751613510:90 , srv761066589:90 , srv880342113:90 , srv890926227:90 , srv997090222:90 }
2023-10-19 02:55:32,845 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=162 (was 296), ProcessCount=167 (was 170), AvailableMemoryMB=10934 (was 10756) - AvailableMemoryMB LEAK? -