2023-07-29 19:53:11,608 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-07-29 19:53:11,925 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-07-29 19:53:12,037 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-07-29 19:53:12,103 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-29 19:53:12,112 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-29 19:53:12,129 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=310, ProcessCount=176, AvailableMemoryMB=2557
2023-07-29 19:53:12,131 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-29 19:53:12,132 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-29 19:53:12,293 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1005232115:84 , srv1020256521:84 , srv1020895033:84 , srv1025339667:84 , srv1035167647:84 , srv1090092356:84 , srv1098216649:84 , srv1120166742:84 , srv1134100524:84 , srv1195429852:84 , srv1253192008:84 , srv1253244110:84 , srv1255739212:84 , srv1257613507:84 , srv1308497174:84 , srv1332719119:84 , srv1339334478:84 , srv1347377823:84 , srv1354509888:84 , srv140526635:84 , srv1416758813:84 , srv1444068164:84 , srv145258667:84 , srv1463488703:84 , srv1480826399:84 , srv1502838009:84 , srv1537571109:84 , srv1537618598:84 , srv1552980909:84 , srv1564321724:84 , srv159109106:84 , srv1605757110:84 , srv1608557957:84 , srv1624543265:84 , srv1626530086:84 , srv1654210960:84 , srv1681259677:84 , srv169333211:84 , srv1715270618:84 , srv1723921963:84 , srv1728105561:84 , srv1730339591:84 , srv1734556265:84 , srv1738233366:84 , srv1758204267:84 , srv1793687290:84 , srv1808813056:84 , srv1820387682:84 , srv1832597687:84 , srv1854875935:84 , srv18652213:84 , srv1895723500:84 , srv1907333972:84 , srv1916123143:84 , srv1949873247:84 , srv197605731:84 , srv1989860664:84 , srv199916775:84 , srv1999310065:84 , srv2002119350:84 , srv2057978585:84 , srv2076102767:84 , srv2079633185:84 , srv2081982630:84 , srv2108813471:84 , srv2112411885:84 , srv2143950047:84 , srv2144884071:84 , srv267627760:84 , srv278428383:84 , srv298723750:84 , srv301437225:84 , srv341799966:84 , srv357717258:84 , srv361688254:84 , srv402051074:84 , srv417625171:84 , srv428484717:84 , srv438007677:84 , srv445323597:84 , srv59713756:84 , srv621692975:84 , srv661921177:84 , srv684286564:84 , srv706193139:84 , srv745188024:84 , srv784360660:84 , srv805562867:84 , srv810281419:84 , srv842796782:84 , srv857616714:84 , srv864070871:84 , srv872078498:84 , srv911132975:84 , srv915609663:84 , srv933296289:84 , srv978763183:84 , srv982000224:84 , srv988091837:84 , srv1802275522:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-07-29 19:53:12,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-29 19:53:12,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv417625171=77, srv933296289=96, srv1195429852=9, srv1035167647=4, srv1552980909=28, srv864070871=92, srv357717258=74, srv988091837=99, srv1907333972=53, srv1916123143=54, srv445323597=80, srv1255739212=12, srv1502838009=25, srv1134100524=8, srv140526635=19, srv1332719119=15, srv978763183=97, srv1339334478=16, srv1734556265=42, srv2108813471=65, srv1989860664=57, srv1537618598=27, srv1854875935=50, srv1308497174=14, srv842796782=90, srv1723921963=39, srv1253192008=10, srv1257613507=13, srv2112411885=66, srv1605757110=31, srv298723750=71, srv2144884071=68, srv1895723500=52, srv1416758813=20, srv857616714=91, srv2002119350=60, srv2057978585=61, srv684286564=84, srv784360660=87, srv1738233366=43, srv361688254=75, srv1999310065=59, srv1463488703=23, srv1444068164=21, srv438007677=79, srv1253244110=11, srv872078498=93, srv145258667=22, srv1025339667=3, srv1347377823=17, srv1802275522=46, srv1730339591=41, srv1793687290=45, srv1758204267=44, srv341799966=73, srv1354509888=18, srv1949873247=55, srv621692975=82, srv1681259677=36, srv1005232115=0, srv1624543265=33, srv1654210960=35, srv2076102767=62, srv267627760=69, srv911132975=94, srv745188024=86, srv661921177=83, srv1715270618=38, srv159109106=30, srv301437225=72, srv2081982630=64, srv199916775=58, srv278428383=70, srv1480826399=24, srv1808813056=47, srv706193139=85, srv1728105561=40, srv2079633185=63, srv1020256521=1, srv1832597687=49, srv805562867=88, srv1098216649=6, srv1564321724=29, srv1820387682=48, srv402051074=76, srv1608557957=32, srv18652213=51, srv1020895033=2, srv1120166742=7, srv197605731=56, srv2143950047=67, srv810281419=89, srv1090092356=5, srv169333211=37, srv428484717=78, srv1537571109=26, srv59713756=81, srv982000224=98, srv1626530086=34, srv915609663=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-29 19:53:12,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-29 19:53:12,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-29 19:53:12,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-29 19:53:12,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-29 19:53:12,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-29 19:53:12,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-29 19:53:12,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-29 19:53:12,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-29 19:53:12,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-29 19:53:12,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-29 19:53:12,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-29 19:53:12,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-29 19:53:12,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-29 19:53:12,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-29 19:53:12,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-29 19:53:12,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-29 19:53:12,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-29 19:53:12,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-29 19:53:12,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-29 19:53:12,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-29 19:53:12,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-29 19:53:12,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-29 19:53:12,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-29 19:53:12,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-29 19:53:12,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-29 19:53:12,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-29 19:53:12,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-29 19:53:12,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-29 19:53:12,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-29 19:53:12,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-29 19:53:12,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-29 19:53:12,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-29 19:53:12,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-29 19:53:12,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-29 19:53:12,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-29 19:53:12,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-29 19:53:12,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-29 19:53:12,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-29 19:53:12,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-29 19:53:12,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-29 19:53:12,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-29 19:53:12,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-29 19:53:12,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-29 19:53:12,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-29 19:53:12,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-29 19:53:12,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-29 19:53:12,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-29 19:53:12,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-29 19:53:12,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-29 19:53:12,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-29 19:53:12,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-29 19:53:12,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-29 19:53:12,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-29 19:53:12,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-29 19:53:12,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-29 19:53:12,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-29 19:53:12,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-29 19:53:12,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-29 19:53:12,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-29 19:53:12,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-29 19:53:12,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-29 19:53:12,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-29 19:53:12,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-29 19:53:12,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-29 19:53:12,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-29 19:53:12,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-29 19:53:12,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-29 19:53:12,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-29 19:53:12,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-29 19:53:12,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-29 19:53:12,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-29 19:53:12,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-29 19:53:12,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-29 19:53:12,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-29 19:53:12,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-29 19:53:12,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-29 19:53:12,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-29 19:53:12,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-29 19:53:12,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-29 19:53:12,939 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:12,940 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-29 19:53:12,941 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-29 19:53:13,413 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 963 ms to try 5000 different iterations.  Found a solution that moves 4658 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18497442980169068. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04043040666090227, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05896983192651203, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5175555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04043040666090227, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18033333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24583333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:13,446 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1444068164:81 , srv2144884071:81 , srv1808813056:82 , srv1895723500:82 , srv784360660:82 , srv864070871:82 , srv988091837:82 , srv1257613507:83 , srv1723921963:83 , srv361688254:83 , srv59713756:83 , srv805562867:83 , srv1120166742:84 , srv1134100524:84 , srv1253244110:84 , srv1347377823:84 , srv1463488703:84 , srv169333211:84 , srv1832597687:84 , srv1854875935:84 , srv1989860664:84 , srv2057978585:84 , srv2108813471:84 , srv267627760:84 , srv301437225:84 , srv341799966:84 , srv402051074:84 , srv445323597:84 , srv745188024:84 , srv1339334478:85 , srv1730339591:85 , srv1734556265:85 , srv197605731:85 , srv1999310065:85 , srv2079633185:85 , srv357717258:85 , srv438007677:85 , srv684286564:85 , srv1005232115:86 , srv1035167647:86 , srv1098216649:86 , srv1332719119:86 , srv140526635:86 , srv1416758813:86 , srv1537618598:86 , srv1624543265:86 , srv1681259677:86 , srv1728105561:86 , srv1820387682:86 , srv18652213:86 , srv1949873247:86 , srv2002119350:86 , srv2081982630:86 , srv2112411885:86 , srv2143950047:86 , srv278428383:86 , srv417625171:86 , srv810281419:86 , srv978763183:86 , srv1020256521:87 , srv1020895033:87 , srv1253192008:87 , srv1255739212:87 , srv1354509888:87 , srv145258667:87 , srv1480826399:87 , srv1793687290:87 , srv298723750:87 , srv621692975:87 , srv1025339667:88 , srv1195429852:88 , srv1552980909:88 , srv1564321724:88 , srv1738233366:88 , srv706193139:88 , srv842796782:88 , srv857616714:88 , srv872078498:88 , srv915609663:88 , srv1502838009:89 , srv1537571109:89 , srv1605757110:89 , srv1654210960:89 , srv428484717:89 , srv661921177:89 , srv1308497174:90 , srv1608557957:90 , srv1626530086:90 , srv1758204267:90 , srv1907333972:90 , srv911132975:90 , srv933296289:90 , srv982000224:90 , srv1090092356:91 , srv159109106:91 , srv1715270618:92 , srv199916775:92 , srv1916123143:93 , srv2076102767:102 , srv1802275522:449 }
2023-07-29 19:53:13,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-29 19:53:13,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv417625171=77, srv933296289=96, srv1195429852=9, srv1035167647=4, srv1552980909=28, srv864070871=92, srv357717258=74, srv988091837=99, srv1907333972=53, srv1916123143=54, srv445323597=80, srv1255739212=12, srv1502838009=25, srv1134100524=8, srv140526635=19, srv1332719119=15, srv978763183=97, srv1339334478=16, srv1734556265=42, srv2108813471=65, srv1989860664=57, srv1537618598=27, srv1854875935=50, srv1308497174=14, srv842796782=90, srv1723921963=39, srv1253192008=10, srv1257613507=13, srv2112411885=66, srv1605757110=31, srv298723750=71, srv2144884071=68, srv1895723500=52, srv1416758813=20, srv857616714=91, srv2002119350=60, srv2057978585=61, srv684286564=84, srv784360660=87, srv1738233366=43, srv361688254=75, srv1999310065=59, srv1463488703=23, srv1444068164=21, srv438007677=79, srv1253244110=11, srv872078498=93, srv145258667=22, srv1025339667=3, srv1347377823=17, srv1802275522=46, srv1730339591=41, srv1793687290=45, srv1758204267=44, srv341799966=73, srv1354509888=18, srv1949873247=55, srv621692975=82, srv1681259677=36, srv1005232115=0, srv1624543265=33, srv1654210960=35, srv2076102767=62, srv267627760=69, srv911132975=94, srv745188024=86, srv661921177=83, srv1715270618=38, srv159109106=30, srv301437225=72, srv2081982630=64, srv199916775=58, srv278428383=70, srv1480826399=24, srv1808813056=47, srv706193139=85, srv1728105561=40, srv2079633185=63, srv1020256521=1, srv1832597687=49, srv805562867=88, srv1098216649=6, srv1564321724=29, srv1820387682=48, srv402051074=76, srv1608557957=32, srv18652213=51, srv1020895033=2, srv1120166742=7, srv197605731=56, srv2143950047=67, srv810281419=89, srv1090092356=5, srv169333211=37, srv428484717=78, srv1537571109=26, srv59713756=81, srv982000224=98, srv1626530086=34, srv915609663=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-29 19:53:13,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-29 19:53:13,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-29 19:53:13,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-29 19:53:13,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-29 19:53:13,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-29 19:53:13,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-29 19:53:13,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-29 19:53:13,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-29 19:53:13,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-29 19:53:13,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-29 19:53:13,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-29 19:53:13,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-29 19:53:13,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-29 19:53:13,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-29 19:53:13,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-29 19:53:13,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-29 19:53:13,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-29 19:53:13,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-29 19:53:13,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-29 19:53:13,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-29 19:53:13,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-29 19:53:13,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-29 19:53:13,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-29 19:53:13,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-29 19:53:13,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-29 19:53:13,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-29 19:53:13,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-29 19:53:13,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-29 19:53:13,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-29 19:53:13,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-29 19:53:13,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-29 19:53:13,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-29 19:53:13,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-29 19:53:13,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-29 19:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-29 19:53:13,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-29 19:53:13,614 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04043040666090227, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05896983192651203, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04043040666090227, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18033333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24583333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:13,614 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-29 19:53:13,615 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18494181378857466, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04043040666090227, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05896983192651203, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04043040666090227, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18033333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24583333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-29 19:53:13,991 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 497 ms to try 5000 different iterations.  Found a solution that moves 2630 regions; Going from a computed imbalance of 0.18494181378857466 to a new imbalance of 0.016152249418091968. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022515461426368757); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03843749572281052, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2922222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022515461426368757); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:14,002 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2076102767:70 , srv1715270618:78 , srv1758204267:79 , srv159109106:80 , srv1608557957:80 , srv199916775:80 , srv911132975:80 , srv1090092356:81 , srv1654210960:81 , srv1626530086:82 , srv1907333972:82 , srv915609663:82 , srv933296289:82 , srv1025339667:83 , srv1308497174:83 , srv1332719119:83 , srv1537571109:83 , srv1605757110:83 , srv1916123143:83 , srv428484717:83 , srv661921177:83 , srv1035167647:84 , srv1195429852:84 , srv1354509888:84 , srv145258667:84 , srv1624543265:84 , srv1728105561:84 , srv1738233366:84 , srv621692975:84 , srv706193139:84 , srv982000224:84 , srv1020256521:85 , srv1255739212:85 , srv140526635:85 , srv1416758813:85 , srv1502838009:85 , srv1552980909:85 , srv1564321724:85 , srv1949873247:85 , srv2079633185:85 , srv2112411885:85 , srv301437225:85 , srv357717258:85 , srv872078498:85 , srv1020895033:86 , srv1480826399:86 , srv1537618598:86 , srv1681259677:86 , srv1793687290:86 , srv18652213:86 , srv2002119350:86 , srv2108813471:86 , srv2143950047:86 , srv361688254:86 , srv438007677:86 , srv857616714:86 , srv1005232115:87 , srv1134100524:87 , srv1253192008:87 , srv1820387682:87 , srv1989860664:87 , srv1999310065:87 , srv2057978585:87 , srv2081982630:87 , srv278428383:87 , srv298723750:87 , srv402051074:87 , srv417625171:87 , srv445323597:87 , srv745188024:87 , srv805562867:87 , srv810281419:87 , srv1098216649:88 , srv1120166742:88 , srv1253244110:88 , srv1339334478:88 , srv1463488703:88 , srv169333211:88 , srv1730339591:88 , srv1832597687:88 , srv197605731:88 , srv267627760:88 , srv59713756:88 , srv684286564:88 , srv978763183:88 , srv988091837:88 , srv1257613507:89 , srv1347377823:89 , srv1734556265:89 , srv1854875935:89 , srv341799966:89 , srv1808813056:90 , srv784360660:90 , srv864070871:90 , srv1444068164:91 , srv1895723500:91 , srv1723921963:92 , srv2144884071:92 , srv842796782:92 , srv1802275522:525 }
2023-07-29 19:53:14,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-29 19:53:14,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv417625171=77, srv933296289=96, srv1195429852=9, srv1035167647=4, srv1552980909=28, srv864070871=92, srv357717258=74, srv988091837=99, srv1907333972=53, srv1916123143=54, srv445323597=80, srv1255739212=12, srv1502838009=25, srv1134100524=8, srv140526635=19, srv1332719119=15, srv978763183=97, srv1339334478=16, srv1734556265=42, srv2108813471=65, srv1989860664=57, srv1537618598=27, srv1854875935=50, srv1308497174=14, srv842796782=90, srv1723921963=39, srv1253192008=10, srv1257613507=13, srv2112411885=66, srv1605757110=31, srv298723750=71, srv2144884071=68, srv1895723500=52, srv1416758813=20, srv857616714=91, srv2002119350=60, srv2057978585=61, srv684286564=84, srv784360660=87, srv1738233366=43, srv361688254=75, srv1999310065=59, srv1463488703=23, srv1444068164=21, srv438007677=79, srv1253244110=11, srv872078498=93, srv145258667=22, srv1025339667=3, srv1347377823=17, srv1802275522=46, srv1730339591=41, srv1793687290=45, srv1758204267=44, srv341799966=73, srv1354509888=18, srv1949873247=55, srv621692975=82, srv1681259677=36, srv1005232115=0, srv1624543265=33, srv1654210960=35, srv2076102767=62, srv267627760=69, srv911132975=94, srv745188024=86, srv661921177=83, srv1715270618=38, srv159109106=30, srv301437225=72, srv2081982630=64, srv199916775=58, srv278428383=70, srv1480826399=24, srv1808813056=47, srv706193139=85, srv1728105561=40, srv2079633185=63, srv1020256521=1, srv1832597687=49, srv805562867=88, srv1098216649=6, srv1564321724=29, srv1820387682=48, srv402051074=76, srv1608557957=32, srv18652213=51, srv1020895033=2, srv1120166742=7, srv197605731=56, srv2143950047=67, srv810281419=89, srv1090092356=5, srv169333211=37, srv428484717=78, srv1537571109=26, srv59713756=81, srv982000224=98, srv1626530086=34, srv915609663=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-29 19:53:14,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-29 19:53:14,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-29 19:53:14,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-29 19:53:14,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-29 19:53:14,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-29 19:53:14,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-29 19:53:14,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-29 19:53:14,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-29 19:53:14,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-29 19:53:14,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-29 19:53:14,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-29 19:53:14,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-29 19:53:14,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-29 19:53:14,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-29 19:53:14,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-29 19:53:14,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-29 19:53:14,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-29 19:53:14,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-29 19:53:14,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-29 19:53:14,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-29 19:53:14,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-29 19:53:14,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-29 19:53:14,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-29 19:53:14,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-29 19:53:14,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-29 19:53:14,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-29 19:53:14,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-29 19:53:14,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-29 19:53:14,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-29 19:53:14,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-29 19:53:14,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-29 19:53:14,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-29 19:53:14,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-29 19:53:14,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-29 19:53:14,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-29 19:53:14,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-29 19:53:14,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-29 19:53:14,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-29 19:53:14,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-29 19:53:14,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-29 19:53:14,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-29 19:53:14,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-29 19:53:14,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-29 19:53:14,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-29 19:53:14,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-29 19:53:14,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-29 19:53:14,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-29 19:53:14,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-29 19:53:14,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-29 19:53:14,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-29 19:53:14,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-29 19:53:14,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-29 19:53:14,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-29 19:53:14,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-29 19:53:14,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-29 19:53:14,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-29 19:53:14,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-29 19:53:14,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-29 19:53:14,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-29 19:53:14,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-29 19:53:14,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-29 19:53:14,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-29 19:53:14,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-29 19:53:14,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-29 19:53:14,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-29 19:53:14,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-29 19:53:14,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-29 19:53:14,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-29 19:53:14,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-29 19:53:14,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-29 19:53:14,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-29 19:53:14,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-29 19:53:14,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-29 19:53:14,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-29 19:53:14,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-29 19:53:14,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-29 19:53:14,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-29 19:53:14,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-29 19:53:14,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-29 19:53:14,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-29 19:53:14,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-29 19:53:14,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-29 19:53:14,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-29 19:53:14,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-29 19:53:14,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-29 19:53:14,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-29 19:53:14,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-29 19:53:14,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-29 19:53:14,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-29 19:53:14,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-29 19:53:14,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-29 19:53:14,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-29 19:53:14,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-29 19:53:14,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-29 19:53:14,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-29 19:53:14,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-29 19:53:14,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-29 19:53:14,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-29 19:53:14,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-29 19:53:14,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-29 19:53:14,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-29 19:53:14,173 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022515461426368757); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03843749572281052, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022515461426368757); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:14,173 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-29 19:53:14,174 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01613383376448631, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022515461426368757); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03843749572281052, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022515461426368757); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-29 19:53:14,483 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 411 ms to try 5000 different iterations.  Found a solution that moves 976 regions; Going from a computed imbalance of 0.01613383376448631 to a new imbalance of 8.147098243000653E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006442160669684254); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012579712168772314); MoveCostFunction : (multiplier=7.0, imbalance=0.10844444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006442160669684254); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:14,486 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv842796782:77 , srv1916123143:82 , srv1253192008:83 , srv1502838009:83 , srv1723921963:83 , srv1730339591:83 , srv1734556265:83 , srv2081982630:83 , srv978763183:83 , srv1005232115:84 , srv1020895033:84 , srv1098216649:84 , srv1308497174:84 , srv1347377823:84 , srv169333211:84 , srv1793687290:84 , srv1854875935:84 , srv1895723500:84 , srv2144884071:84 , srv278428383:84 , srv684286564:84 , srv810281419:84 , srv857616714:84 , srv982000224:84 , srv1020256521:85 , srv1120166742:85 , srv1253244110:85 , srv1339334478:85 , srv1444068164:85 , srv1537571109:85 , srv1537618598:85 , srv1552980909:85 , srv1564321724:85 , srv1626530086:85 , srv1681259677:85 , srv1808813056:85 , srv1820387682:85 , srv1832597687:85 , srv1907333972:85 , srv197605731:85 , srv1999310065:85 , srv2076102767:85 , srv2143950047:85 , srv298723750:85 , srv341799966:85 , srv402051074:85 , srv417625171:85 , srv661921177:85 , srv706193139:85 , srv784360660:85 , srv864070871:85 , srv872078498:85 , srv933296289:85 , srv1025339667:86 , srv1090092356:86 , srv1134100524:86 , srv1195429852:86 , srv1255739212:86 , srv1257613507:86 , srv1354509888:86 , srv1416758813:86 , srv1463488703:86 , srv159109106:86 , srv1605757110:86 , srv1738233366:86 , srv199916775:86 , srv2002119350:86 , srv2057978585:86 , srv267627760:86 , srv428484717:86 , srv438007677:86 , srv59713756:86 , srv621692975:86 , srv140526635:87 , srv145258667:87 , srv1480826399:87 , srv1624543265:87 , srv1715270618:87 , srv1728105561:87 , srv18652213:87 , srv1949873247:87 , srv1989860664:87 , srv2079633185:87 , srv2108813471:87 , srv2112411885:87 , srv445323597:87 , srv745188024:87 , srv805562867:87 , srv911132975:87 , srv915609663:87 , srv988091837:87 , srv1332719119:88 , srv1608557957:88 , srv1758204267:88 , srv357717258:88 , srv1035167647:89 , srv1654210960:89 , srv301437225:89 , srv361688254:89 , srv1802275522:541 }
2023-07-29 19:53:14,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-29 19:53:14,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv417625171=77, srv933296289=96, srv1195429852=9, srv1035167647=4, srv1552980909=28, srv864070871=92, srv357717258=74, srv988091837=99, srv1907333972=53, srv1916123143=54, srv445323597=80, srv1255739212=12, srv1502838009=25, srv1134100524=8, srv140526635=19, srv1332719119=15, srv978763183=97, srv1339334478=16, srv1734556265=42, srv2108813471=65, srv1989860664=57, srv1537618598=27, srv1854875935=50, srv1308497174=14, srv842796782=90, srv1723921963=39, srv1253192008=10, srv1257613507=13, srv2112411885=66, srv1605757110=31, srv298723750=71, srv2144884071=68, srv1895723500=52, srv1416758813=20, srv857616714=91, srv2002119350=60, srv2057978585=61, srv684286564=84, srv784360660=87, srv1738233366=43, srv361688254=75, srv1999310065=59, srv1463488703=23, srv1444068164=21, srv438007677=79, srv1253244110=11, srv872078498=93, srv145258667=22, srv1025339667=3, srv1347377823=17, srv1802275522=46, srv1730339591=41, srv1793687290=45, srv1758204267=44, srv341799966=73, srv1354509888=18, srv1949873247=55, srv621692975=82, srv1681259677=36, srv1005232115=0, srv1624543265=33, srv1654210960=35, srv2076102767=62, srv267627760=69, srv911132975=94, srv745188024=86, srv661921177=83, srv1715270618=38, srv159109106=30, srv301437225=72, srv2081982630=64, srv199916775=58, srv278428383=70, srv1480826399=24, srv1808813056=47, srv706193139=85, srv1728105561=40, srv2079633185=63, srv1020256521=1, srv1832597687=49, srv805562867=88, srv1098216649=6, srv1564321724=29, srv1820387682=48, srv402051074=76, srv1608557957=32, srv18652213=51, srv1020895033=2, srv1120166742=7, srv197605731=56, srv2143950047=67, srv810281419=89, srv1090092356=5, srv169333211=37, srv428484717=78, srv1537571109=26, srv59713756=81, srv982000224=98, srv1626530086=34, srv915609663=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-29 19:53:14,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-29 19:53:14,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-29 19:53:14,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-29 19:53:14,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-29 19:53:14,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-29 19:53:14,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-29 19:53:14,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-29 19:53:14,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-29 19:53:14,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-29 19:53:14,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-29 19:53:14,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-29 19:53:14,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-29 19:53:14,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-29 19:53:14,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-29 19:53:14,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-29 19:53:14,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-29 19:53:14,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-29 19:53:14,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-29 19:53:14,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-29 19:53:14,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-29 19:53:14,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-29 19:53:14,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-29 19:53:14,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-29 19:53:14,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-29 19:53:14,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-29 19:53:14,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-29 19:53:14,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-29 19:53:14,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-29 19:53:14,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-29 19:53:14,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-29 19:53:14,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-29 19:53:14,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-29 19:53:14,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-29 19:53:14,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-29 19:53:14,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-29 19:53:14,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-29 19:53:14,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-29 19:53:14,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-29 19:53:14,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-29 19:53:14,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-29 19:53:14,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-29 19:53:14,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-29 19:53:14,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-29 19:53:14,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-29 19:53:14,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-29 19:53:14,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-29 19:53:14,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-29 19:53:14,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-29 19:53:14,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-29 19:53:14,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-29 19:53:14,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-29 19:53:14,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-29 19:53:14,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-29 19:53:14,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-29 19:53:14,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-29 19:53:14,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-29 19:53:14,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-29 19:53:14,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-29 19:53:14,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-29 19:53:14,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-29 19:53:14,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-29 19:53:14,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-29 19:53:14,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-29 19:53:14,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-29 19:53:14,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-29 19:53:14,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-29 19:53:14,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-29 19:53:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-29 19:53:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-29 19:53:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-29 19:53:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-29 19:53:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-29 19:53:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-29 19:53:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-29 19:53:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-29 19:53:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-29 19:53:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-29 19:53:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-29 19:53:14,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-29 19:53:14,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-29 19:53:14,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-29 19:53:14,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-29 19:53:14,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-29 19:53:14,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-29 19:53:14,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-29 19:53:14,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-29 19:53:14,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-29 19:53:14,672 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006442160669684254); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012579712168772314); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006442160669684254); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:14,672 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-29 19:53:14,672 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.078757262319584E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006442160669684254); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012579712168772314); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006442160669684254); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-29 19:53:15,102 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 565 ms to try 5000 different iterations.  Found a solution that moves 263 regions; Going from a computed imbalance of 8.078757262319584E-4 to a new imbalance of 3.4259021164489484E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.029222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:15,103 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1035167647:83 , srv145258667:83 , srv1480826399:83 , srv1654210960:83 , srv1738233366:83 , srv18652213:83 , srv2112411885:83 , srv1090092356:84 , srv1098216649:84 , srv1120166742:84 , srv1253192008:84 , srv1255739212:84 , srv1257613507:84 , srv1339334478:84 , srv140526635:84 , srv1463488703:84 , srv1552980909:84 , srv1564321724:84 , srv1605757110:84 , srv1608557957:84 , srv1758204267:84 , srv1820387682:84 , srv1854875935:84 , srv1895723500:84 , srv1916123143:84 , srv1949873247:84 , srv197605731:84 , srv1989860664:84 , srv2002119350:84 , srv2076102767:84 , srv298723750:84 , srv341799966:84 , srv357717258:84 , srv361688254:84 , srv417625171:84 , srv428484717:84 , srv445323597:84 , srv621692975:84 , srv745188024:84 , srv857616714:84 , srv911132975:84 , srv982000224:84 , srv1005232115:85 , srv1020256521:85 , srv1020895033:85 , srv1025339667:85 , srv1134100524:85 , srv1195429852:85 , srv1253244110:85 , srv1308497174:85 , srv1332719119:85 , srv1347377823:85 , srv1354509888:85 , srv1416758813:85 , srv1502838009:85 , srv1537571109:85 , srv1537618598:85 , srv159109106:85 , srv1624543265:85 , srv1626530086:85 , srv1681259677:85 , srv1715270618:85 , srv1723921963:85 , srv1728105561:85 , srv1734556265:85 , srv1808813056:85 , srv1832597687:85 , srv1907333972:85 , srv199916775:85 , srv2057978585:85 , srv2079633185:85 , srv2108813471:85 , srv2143950047:85 , srv2144884071:85 , srv267627760:85 , srv278428383:85 , srv301437225:85 , srv438007677:85 , srv59713756:85 , srv661921177:85 , srv684286564:85 , srv706193139:85 , srv784360660:85 , srv805562867:85 , srv810281419:85 , srv842796782:85 , srv864070871:85 , srv872078498:85 , srv915609663:85 , srv933296289:85 , srv978763183:85 , srv988091837:85 , srv1444068164:86 , srv169333211:86 , srv1730339591:86 , srv1793687290:86 , srv1999310065:86 , srv2081982630:86 , srv402051074:86 , srv1802275522:627 }
2023-07-29 19:53:15,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-29 19:53:15,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv417625171=77, srv933296289=96, srv1195429852=9, srv1035167647=4, srv1552980909=28, srv864070871=92, srv357717258=74, srv988091837=99, srv1907333972=53, srv1916123143=54, srv445323597=80, srv1255739212=12, srv1502838009=25, srv1134100524=8, srv140526635=19, srv1332719119=15, srv978763183=97, srv1339334478=16, srv1734556265=42, srv2108813471=65, srv1989860664=57, srv1537618598=27, srv1854875935=50, srv1308497174=14, srv842796782=90, srv1723921963=39, srv1253192008=10, srv1257613507=13, srv2112411885=66, srv1605757110=31, srv298723750=71, srv2144884071=68, srv1895723500=52, srv1416758813=20, srv857616714=91, srv2002119350=60, srv2057978585=61, srv684286564=84, srv784360660=87, srv1738233366=43, srv361688254=75, srv1999310065=59, srv1463488703=23, srv1444068164=21, srv438007677=79, srv1253244110=11, srv872078498=93, srv145258667=22, srv1025339667=3, srv1347377823=17, srv1802275522=46, srv1730339591=41, srv1793687290=45, srv1758204267=44, srv341799966=73, srv1354509888=18, srv1949873247=55, srv621692975=82, srv1681259677=36, srv1005232115=0, srv1624543265=33, srv1654210960=35, srv2076102767=62, srv267627760=69, srv911132975=94, srv745188024=86, srv661921177=83, srv1715270618=38, srv159109106=30, srv301437225=72, srv2081982630=64, srv199916775=58, srv278428383=70, srv1480826399=24, srv1808813056=47, srv706193139=85, srv1728105561=40, srv2079633185=63, srv1020256521=1, srv1832597687=49, srv805562867=88, srv1098216649=6, srv1564321724=29, srv1820387682=48, srv402051074=76, srv1608557957=32, srv18652213=51, srv1020895033=2, srv1120166742=7, srv197605731=56, srv2143950047=67, srv810281419=89, srv1090092356=5, srv169333211=37, srv428484717=78, srv1537571109=26, srv59713756=81, srv982000224=98, srv1626530086=34, srv915609663=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-29 19:53:15,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-29 19:53:15,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-29 19:53:15,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-29 19:53:15,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-29 19:53:15,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-29 19:53:15,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-29 19:53:15,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-29 19:53:15,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-29 19:53:15,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-29 19:53:15,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-29 19:53:15,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-29 19:53:15,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-29 19:53:15,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-29 19:53:15,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-29 19:53:15,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-29 19:53:15,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-29 19:53:15,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-29 19:53:15,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-29 19:53:15,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-29 19:53:15,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-29 19:53:15,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-29 19:53:15,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-29 19:53:15,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-29 19:53:15,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-29 19:53:15,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-29 19:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-29 19:53:15,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-29 19:53:15,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-29 19:53:15,243 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:15,243 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-29 19:53:15,243 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.2417455803923784E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-29 19:53:15,537 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 394 ms to try 5000 different iterations.  Found a solution that moves 23 regions; Going from a computed imbalance of 3.2417455803923784E-5 to a new imbalance of 6.5590351144484475E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0025555555555555557); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:15,538 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1195429852:83 , srv1444068164:83 , srv1480826399:83 , srv1723921963:83 , srv1793687290:83 , srv199916775:83 , srv1999310065:83 , srv267627760:83 , srv298723750:83 , srv301437225:83 , srv872078498:83 , srv1005232115:84 , srv1020256521:84 , srv1020895033:84 , srv1025339667:84 , srv1035167647:84 , srv1090092356:84 , srv1098216649:84 , srv1134100524:84 , srv1253244110:84 , srv1255739212:84 , srv1257613507:84 , srv1308497174:84 , srv1332719119:84 , srv1339334478:84 , srv1347377823:84 , srv1354509888:84 , srv140526635:84 , srv1416758813:84 , srv1463488703:84 , srv1502838009:84 , srv1537571109:84 , srv1537618598:84 , srv1552980909:84 , srv1564321724:84 , srv159109106:84 , srv1605757110:84 , srv1608557957:84 , srv1624543265:84 , srv1626530086:84 , srv1654210960:84 , srv1681259677:84 , srv169333211:84 , srv1715270618:84 , srv1728105561:84 , srv1730339591:84 , srv1734556265:84 , srv1808813056:84 , srv1820387682:84 , srv1832597687:84 , srv18652213:84 , srv1907333972:84 , srv1916123143:84 , srv1949873247:84 , srv197605731:84 , srv1989860664:84 , srv2002119350:84 , srv2057978585:84 , srv2079633185:84 , srv2081982630:84 , srv2108813471:84 , srv2143950047:84 , srv2144884071:84 , srv278428383:84 , srv341799966:84 , srv357717258:84 , srv361688254:84 , srv402051074:84 , srv417625171:84 , srv428484717:84 , srv438007677:84 , srv445323597:84 , srv59713756:84 , srv661921177:84 , srv684286564:84 , srv706193139:84 , srv745188024:84 , srv784360660:84 , srv805562867:84 , srv810281419:84 , srv842796782:84 , srv857616714:84 , srv864070871:84 , srv915609663:84 , srv933296289:84 , srv978763183:84 , srv982000224:84 , srv988091837:84 , srv1120166742:85 , srv1253192008:85 , srv145258667:85 , srv1738233366:85 , srv1758204267:85 , srv1854875935:85 , srv1895723500:85 , srv2076102767:85 , srv2112411885:85 , srv621692975:85 , srv911132975:85 , srv1802275522:684 }
2023-07-29 19:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-29 19:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv417625171=77, srv933296289=96, srv1195429852=9, srv1035167647=4, srv1552980909=28, srv864070871=92, srv357717258=74, srv988091837=99, srv1907333972=53, srv1916123143=54, srv445323597=80, srv1255739212=12, srv1502838009=25, srv1134100524=8, srv140526635=19, srv1332719119=15, srv978763183=97, srv1339334478=16, srv1734556265=42, srv2108813471=65, srv1989860664=57, srv1537618598=27, srv1854875935=50, srv1308497174=14, srv842796782=90, srv1723921963=39, srv1253192008=10, srv1257613507=13, srv2112411885=66, srv1605757110=31, srv298723750=71, srv2144884071=68, srv1895723500=52, srv1416758813=20, srv857616714=91, srv2002119350=60, srv2057978585=61, srv684286564=84, srv784360660=87, srv1738233366=43, srv361688254=75, srv1999310065=59, srv1463488703=23, srv1444068164=21, srv438007677=79, srv1253244110=11, srv872078498=93, srv145258667=22, srv1025339667=3, srv1347377823=17, srv1802275522=46, srv1730339591=41, srv1793687290=45, srv1758204267=44, srv341799966=73, srv1354509888=18, srv1949873247=55, srv621692975=82, srv1681259677=36, srv1005232115=0, srv1624543265=33, srv1654210960=35, srv2076102767=62, srv267627760=69, srv911132975=94, srv745188024=86, srv661921177=83, srv1715270618=38, srv159109106=30, srv301437225=72, srv2081982630=64, srv199916775=58, srv278428383=70, srv1480826399=24, srv1808813056=47, srv706193139=85, srv1728105561=40, srv2079633185=63, srv1020256521=1, srv1832597687=49, srv805562867=88, srv1098216649=6, srv1564321724=29, srv1820387682=48, srv402051074=76, srv1608557957=32, srv18652213=51, srv1020895033=2, srv1120166742=7, srv197605731=56, srv2143950047=67, srv810281419=89, srv1090092356=5, srv169333211=37, srv428484717=78, srv1537571109=26, srv59713756=81, srv982000224=98, srv1626530086=34, srv915609663=95} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-29 19:53:15,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-29 19:53:15,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-29 19:53:15,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-29 19:53:15,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-29 19:53:15,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-29 19:53:15,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-29 19:53:15,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-29 19:53:15,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-29 19:53:15,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-29 19:53:15,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-29 19:53:15,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-29 19:53:15,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-29 19:53:15,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-29 19:53:15,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-29 19:53:15,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-29 19:53:15,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-29 19:53:15,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-29 19:53:15,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-29 19:53:15,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-29 19:53:15,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-29 19:53:15,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-29 19:53:15,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-29 19:53:15,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-29 19:53:15,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-29 19:53:15,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-29 19:53:15,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-29 19:53:15,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-29 19:53:15,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-29 19:53:15,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-29 19:53:15,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-29 19:53:15,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-29 19:53:15,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-29 19:53:15,690 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:15,690 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1195429852:83 , srv1444068164:83 , srv1480826399:83 , srv1723921963:83 , srv1793687290:83 , srv199916775:83 , srv1999310065:83 , srv267627760:83 , srv298723750:83 , srv301437225:83 , srv872078498:83 , srv1005232115:84 , srv1020256521:84 , srv1020895033:84 , srv1025339667:84 , srv1035167647:84 , srv1090092356:84 , srv1098216649:84 , srv1134100524:84 , srv1253244110:84 , srv1255739212:84 , srv1257613507:84 , srv1308497174:84 , srv1332719119:84 , srv1339334478:84 , srv1347377823:84 , srv1354509888:84 , srv140526635:84 , srv1416758813:84 , srv1463488703:84 , srv1502838009:84 , srv1537571109:84 , srv1537618598:84 , srv1552980909:84 , srv1564321724:84 , srv159109106:84 , srv1605757110:84 , srv1608557957:84 , srv1624543265:84 , srv1626530086:84 , srv1654210960:84 , srv1681259677:84 , srv169333211:84 , srv1715270618:84 , srv1728105561:84 , srv1730339591:84 , srv1734556265:84 , srv1808813056:84 , srv1820387682:84 , srv1832597687:84 , srv18652213:84 , srv1907333972:84 , srv1916123143:84 , srv1949873247:84 , srv197605731:84 , srv1989860664:84 , srv2002119350:84 , srv2057978585:84 , srv2079633185:84 , srv2081982630:84 , srv2108813471:84 , srv2143950047:84 , srv2144884071:84 , srv278428383:84 , srv341799966:84 , srv357717258:84 , srv361688254:84 , srv402051074:84 , srv417625171:84 , srv428484717:84 , srv438007677:84 , srv445323597:84 , srv59713756:84 , srv661921177:84 , srv684286564:84 , srv706193139:84 , srv745188024:84 , srv784360660:84 , srv805562867:84 , srv810281419:84 , srv842796782:84 , srv857616714:84 , srv864070871:84 , srv915609663:84 , srv933296289:84 , srv978763183:84 , srv982000224:84 , srv988091837:84 , srv1120166742:85 , srv1253192008:85 , srv145258667:85 , srv1738233366:85 , srv1758204267:85 , srv1854875935:85 , srv1895723500:85 , srv2076102767:85 , srv2112411885:85 , srv621692975:85 , srv911132975:85 , srv1802275522:684 }
2023-07-29 19:53:15,723 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=333 (was 310) - SystemLoadAverage LEAK? -, ProcessCount=176 (was 176), AvailableMemoryMB=1701 (was 2557)
2023-07-29 19:53:15,732 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=333, ProcessCount=176, AvailableMemoryMB=1701
2023-07-29 19:53:15,733 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-29 19:53:15,733 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-07-29 19:53:15,746 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1011182518:84 , srv1076572666:84 , srv1283865384:84 , srv1468977016:84 , srv1473621468:84 , srv1567795224:84 , srv1644265832:84 , srv1673279770:84 , srv1682314665:84 , srv1687020355:84 , srv1709822498:84 , srv1963960804:84 , srv199411689:84 , srv2020485581:84 , srv2049979603:84 , srv2136087285:84 , srv258409916:84 , srv321472146:84 , srv373347195:84 , srv408188539:84 , srv428131704:84 , srv476986362:84 , srv575631518:84 , srv59908994:84 , srv608070137:84 , srv637392176:84 , srv69945941:84 , srv789845004:84 , srv831381776:84 , srv2089534134:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-07-29 19:53:15,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-29 19:53:15,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv789845004=28, srv575631518=23, srv1473621468=4, srv637392176=26, srv1709822498=10, srv2049979603=14, srv2020485581=13, srv69945941=27, srv373347195=19, srv1687020355=9, srv321472146=18, srv199411689=12, srv2089534134=15, srv608070137=25, srv1963960804=11, srv1076572666=1, srv476986362=22, srv1682314665=8, srv1468977016=3, srv1283865384=2, srv1673279770=7, srv1644265832=6, srv408188539=20, srv258409916=17, srv2136087285=16, srv59908994=24, srv428131704=21, srv1567795224=5, srv1011182518=0, srv831381776=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-29 19:53:15,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-29 19:53:15,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-29 19:53:15,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-29 19:53:15,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-29 19:53:15,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-29 19:53:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-29 19:53:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-29 19:53:15,781 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.163651032789419, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:53:15,781 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912708702324899, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.163651032789419, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-07-29 19:55:15,766 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5632584 different iterations.  Found a solution that moves 1877 regions; Going from a computed imbalance of 0.9912708702324899 to a new imbalance of 4.3810116372393E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6951851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:55:15,773 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1011182518:90 , srv1076572666:90 , srv1283865384:90 , srv1468977016:90 , srv1473621468:90 , srv1567795224:90 , srv1644265832:90 , srv1673279770:90 , srv1682314665:90 , srv1687020355:90 , srv1709822498:90 , srv1963960804:90 , srv199411689:90 , srv2020485581:90 , srv2049979603:90 , srv2089534134:90 , srv2136087285:90 , srv258409916:90 , srv321472146:90 , srv373347195:90 , srv408188539:90 , srv428131704:90 , srv476986362:90 , srv575631518:90 , srv59908994:90 , srv608070137:90 , srv637392176:90 , srv69945941:90 , srv789845004:90 , srv831381776:90 }
2023-07-29 19:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-29 19:55:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv789845004=28, srv575631518=23, srv1473621468=4, srv637392176=26, srv1709822498=10, srv2049979603=14, srv2020485581=13, srv69945941=27, srv373347195=19, srv1687020355=9, srv321472146=18, srv199411689=12, srv2089534134=15, srv608070137=25, srv1963960804=11, srv1076572666=1, srv476986362=22, srv1682314665=8, srv1468977016=3, srv1283865384=2, srv1673279770=7, srv1644265832=6, srv408188539=20, srv258409916=17, srv2136087285=16, srv59908994=24, srv428131704=21, srv1567795224=5, srv1011182518=0, srv831381776=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-29 19:55:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-29 19:55:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-29 19:55:15,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-29 19:55:15,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-29 19:55:15,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-29 19:55:15,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-29 19:55:15,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-29 19:55:15,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-29 19:55:15,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-29 19:55:15,792 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-29 19:55:15,792 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1011182518:90 , srv1076572666:90 , srv1283865384:90 , srv1468977016:90 , srv1473621468:90 , srv1567795224:90 , srv1644265832:90 , srv1673279770:90 , srv1682314665:90 , srv1687020355:90 , srv1709822498:90 , srv1963960804:90 , srv199411689:90 , srv2020485581:90 , srv2049979603:90 , srv2089534134:90 , srv2136087285:90 , srv258409916:90 , srv321472146:90 , srv373347195:90 , srv408188539:90 , srv428131704:90 , srv476986362:90 , srv575631518:90 , srv59908994:90 , srv608070137:90 , srv637392176:90 , srv69945941:90 , srv789845004:90 , srv831381776:90 }
2023-07-29 19:55:15,808 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=212 (was 333), ProcessCount=172 (was 176), AvailableMemoryMB=1653 (was 1701)