2023-03-23 10:56:04,557 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-23 10:56:04,869 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-23 10:56:04,974 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-23 10:56:05,035 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-23 10:56:05,042 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-23 10:56:05,055 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=355, ProcessCount=169, AvailableMemoryMB=6523
2023-03-23 10:56:05,057 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-23 10:56:05,058 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-23 10:56:05,184 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000838094:84 , srv1008460674:84 , srv1049895207:84 , srv1059189728:84 , srv1063519069:84 , srv1129301054:84 , srv1178442261:84 , srv1182663925:84 , srv1220106377:84 , srv1226285208:84 , srv1259100093:84 , srv12734738:84 , srv1290883419:84 , srv1300701442:84 , srv1300824279:84 , srv1310948640:84 , srv1327925253:84 , srv1377904360:84 , srv143861193:84 , srv1450234880:84 , srv146933674:84 , srv1516989518:84 , srv1521351696:84 , srv1531321469:84 , srv1534579809:84 , srv1539212680:84 , srv156309816:84 , srv1608684249:84 , srv1650903498:84 , srv1672230323:84 , srv1675101671:84 , srv1675539738:84 , srv1731164529:84 , srv1732344039:84 , srv1734678055:84 , srv1756724145:84 , srv1774341004:84 , srv1783803381:84 , srv1794285146:84 , srv1805742701:84 , srv1824363472:84 , srv1825981957:84 , srv1827864715:84 , srv1830095843:84 , srv1839495433:84 , srv1843235609:84 , srv1843329058:84 , srv1848901335:84 , srv18544311:84 , srv1856395132:84 , srv1862216741:84 , srv1863544153:84 , srv1923345559:84 , srv1941878633:84 , srv1958490516:84 , srv1969291383:84 , srv1988648884:84 , srv1993950814:84 , srv2018140672:84 , srv2050854110:84 , srv205115543:84 , srv2054949915:84 , srv2072752412:84 , srv2106106884:84 , srv2114456759:84 , srv2116081636:84 , srv2120410572:84 , srv2143241326:84 , srv235928878:84 , srv272398677:84 , srv328750537:84 , srv356049607:84 , srv372901877:84 , srv411648640:84 , srv476794657:84 , srv505932677:84 , srv508186383:84 , srv511815527:84 , srv534742078:84 , srv541450037:84 , srv559713020:84 , srv579412003:84 , srv587728494:84 , srv61356507:84 , srv621308985:84 , srv641156857:84 , srv652185072:84 , srv652769146:84 , srv692373354:84 , srv713484207:84 , srv724971295:84 , srv772164310:84 , srv885695930:84 , srv909396592:84 , srv914607579:84 , srv936747551:84 , srv968589837:84 , srv991170354:84 , srv996088749:84 , srv995944291:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-23 10:56:05,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 10:56:05,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv272398677=69, srv1178442261=6, srv18544311=48, srv772164310=91, srv1839495433=44, srv1521351696=22, srv2114456759=64, srv1049895207=2, srv2072752412=62, srv505932677=75, srv652185072=86, srv1794285146=38, srv1516989518=21, srv1805742701=39, srv1675539738=31, srv1650903498=28, srv1862216741=50, srv995944291=98, srv2106106884=63, srv476794657=74, srv1958490516=54, srv61356507=83, srv1182663925=7, srv143861193=18, srv2050854110=59, srv508186383=76, srv1756724145=35, srv1672230323=29, srv1941878633=53, srv411648640=73, srv587728494=82, srv713484207=89, srv1531321469=23, srv1675101671=30, srv1220106377=8, srv1310948640=15, srv1608684249=27, srv936747551=95, srv885695930=92, srv2143241326=67, srv1059189728=3, srv1988648884=56, srv1830095843=43, srv968589837=96, srv541450037=79, srv1863544153=51, srv621308985=84, srv1856395132=49, srv914607579=94, srv1923345559=52, srv652769146=87, srv1843329058=46, srv12734738=11, srv909396592=93, srv1226285208=9, srv1300701442=13, srv2054949915=61, srv1290883419=12, srv1377904360=17, srv534742078=78, srv146933674=20, srv1450234880=19, srv1731164529=32, srv724971295=90, srv2120410572=66, srv1129301054=5, srv1827864715=42, srv1993950814=57, srv511815527=77, srv1825981957=41, srv205115543=60, srv2116081636=65, srv1969291383=55, srv641156857=85, srv1259100093=10, srv328750537=70, srv1824363472=40, srv1000838094=0, srv559713020=80, srv1534579809=24, srv1732344039=33, srv996088749=99, srv156309816=26, srv1327925253=16, srv356049607=71, srv1734678055=34, srv1008460674=1, srv1783803381=37, srv1063519069=4, srv991170354=97, srv1539212680=25, srv372901877=72, srv1300824279=14, srv579412003=81, srv692373354=88, srv1843235609=45, srv235928878=68, srv2018140672=58, srv1774341004=36, srv1848901335=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 10:56:05,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 10:56:05,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 10:56:05,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 10:56:05,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 10:56:05,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 10:56:05,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 10:56:05,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 10:56:05,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 10:56:05,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 10:56:05,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 10:56:05,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 10:56:05,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 10:56:05,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 10:56:05,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 10:56:05,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 10:56:05,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 10:56:05,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 10:56:05,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 10:56:05,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 10:56:05,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 10:56:05,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 10:56:05,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 10:56:05,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 10:56:05,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 10:56:05,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 10:56:05,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 10:56:05,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 10:56:05,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 10:56:05,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 10:56:05,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 10:56:05,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 10:56:05,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 10:56:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 10:56:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 10:56:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 10:56:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 10:56:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 10:56:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 10:56:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 10:56:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 10:56:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 10:56:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 10:56:05,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 10:56:05,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 10:56:05,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 10:56:05,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 10:56:05,502 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:05,502 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 10:56:05,502 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 10:56:06,115 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 788 ms to try 5000 different iterations.  Found a solution that moves 4713 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17928963445393672. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04129014312993761, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05673099933081517, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5236666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04129014312993761, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24191666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:06,136 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv559713020:79 , srv1843235609:81 , srv356049607:81 , srv146933674:82 , srv272398677:82 , srv587728494:82 , srv1008460674:83 , srv143861193:83 , srv156309816:83 , srv1672230323:83 , srv2116081636:83 , srv372901877:83 , srv411648640:83 , srv511815527:83 , srv652769146:83 , srv1290883419:84 , srv1300701442:84 , srv1310948640:84 , srv1516989518:84 , srv18544311:84 , srv2054949915:84 , srv2120410572:84 , srv2143241326:84 , srv12734738:85 , srv1450234880:85 , srv1539212680:85 , srv1731164529:85 , srv1862216741:85 , srv205115543:85 , srv534742078:85 , srv579412003:85 , srv641156857:85 , srv692373354:85 , srv713484207:85 , srv996088749:85 , srv1327925253:86 , srv1531321469:86 , srv1608684249:86 , srv1805742701:86 , srv1825981957:86 , srv1848901335:86 , srv1856395132:86 , srv1863544153:86 , srv1923345559:86 , srv1958490516:86 , srv1988648884:86 , srv1993950814:86 , srv2050854110:86 , srv2106106884:86 , srv476794657:86 , srv885695930:86 , srv968589837:86 , srv991170354:86 , srv1129301054:87 , srv1220106377:87 , srv1226285208:87 , srv1259100093:87 , srv1300824279:87 , srv1377904360:87 , srv1675101671:87 , srv1732344039:87 , srv1734678055:87 , srv1774341004:87 , srv1827864715:87 , srv1839495433:87 , srv2018140672:87 , srv2072752412:87 , srv235928878:87 , srv914607579:87 , srv936747551:87 , srv1000838094:88 , srv1063519069:88 , srv1178442261:88 , srv1521351696:88 , srv1756724145:88 , srv1783803381:88 , srv1830095843:88 , srv1843329058:88 , srv1941878633:88 , srv508186383:88 , srv541450037:88 , srv1059189728:89 , srv1182663925:89 , srv1650903498:89 , srv1675539738:89 , srv1824363472:89 , srv2114456759:89 , srv328750537:89 , srv505932677:89 , srv621308985:89 , srv909396592:89 , srv1534579809:90 , srv1794285146:91 , srv1969291383:91 , srv61356507:91 , srv652185072:91 , srv724971295:92 , srv1049895207:93 , srv772164310:93 , srv995944291:457 }
2023-03-23 10:56:06,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 10:56:06,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv272398677=69, srv1178442261=6, srv18544311=48, srv772164310=91, srv1839495433=44, srv1521351696=22, srv2114456759=64, srv1049895207=2, srv2072752412=62, srv505932677=75, srv652185072=86, srv1794285146=38, srv1516989518=21, srv1805742701=39, srv1675539738=31, srv1650903498=28, srv1862216741=50, srv995944291=98, srv2106106884=63, srv476794657=74, srv1958490516=54, srv61356507=83, srv1182663925=7, srv143861193=18, srv2050854110=59, srv508186383=76, srv1756724145=35, srv1672230323=29, srv1941878633=53, srv411648640=73, srv587728494=82, srv713484207=89, srv1531321469=23, srv1675101671=30, srv1220106377=8, srv1310948640=15, srv1608684249=27, srv936747551=95, srv885695930=92, srv2143241326=67, srv1059189728=3, srv1988648884=56, srv1830095843=43, srv968589837=96, srv541450037=79, srv1863544153=51, srv621308985=84, srv1856395132=49, srv914607579=94, srv1923345559=52, srv652769146=87, srv1843329058=46, srv12734738=11, srv909396592=93, srv1226285208=9, srv1300701442=13, srv2054949915=61, srv1290883419=12, srv1377904360=17, srv534742078=78, srv146933674=20, srv1450234880=19, srv1731164529=32, srv724971295=90, srv2120410572=66, srv1129301054=5, srv1827864715=42, srv1993950814=57, srv511815527=77, srv1825981957=41, srv205115543=60, srv2116081636=65, srv1969291383=55, srv641156857=85, srv1259100093=10, srv328750537=70, srv1824363472=40, srv1000838094=0, srv559713020=80, srv1534579809=24, srv1732344039=33, srv996088749=99, srv156309816=26, srv1327925253=16, srv356049607=71, srv1734678055=34, srv1008460674=1, srv1783803381=37, srv1063519069=4, srv991170354=97, srv1539212680=25, srv372901877=72, srv1300824279=14, srv579412003=81, srv692373354=88, srv1843235609=45, srv235928878=68, srv2018140672=58, srv1774341004=36, srv1848901335=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 10:56:06,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 10:56:06,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 10:56:06,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 10:56:06,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 10:56:06,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 10:56:06,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 10:56:06,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 10:56:06,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 10:56:06,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 10:56:06,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 10:56:06,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 10:56:06,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 10:56:06,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 10:56:06,318 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04129014312993761, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05673099933081517, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04129014312993761, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24191666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:06,319 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 10:56:06,319 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17925663332258937, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04129014312993761, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05673099933081517, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04129014312993761, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24191666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 10:56:06,610 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 423 ms to try 5000 different iterations.  Found a solution that moves 2663 regions; Going from a computed imbalance of 0.17925663332258937 to a new imbalance of 0.015930598805453566. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021600737125608054); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033457680524552995, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29588888888888887, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021600737125608054); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:06,620 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1049895207:79 , srv1969291383:80 , srv772164310:80 , srv1059189728:81 , srv1534579809:81 , srv1794285146:81 , srv724971295:81 , srv1063519069:82 , srv1843329058:82 , srv505932677:82 , srv61356507:82 , srv621308985:82 , srv1000838094:83 , srv1521351696:83 , srv1675539738:83 , srv1756724145:83 , srv2114456759:83 , srv328750537:83 , srv968589837:83 , srv1734678055:84 , srv1783803381:84 , srv1824363472:84 , srv1830095843:84 , srv1863544153:84 , srv1958490516:84 , srv1993950814:84 , srv235928878:84 , srv508186383:84 , srv541450037:84 , srv652185072:84 , srv1129301054:85 , srv1377904360:85 , srv1531321469:85 , srv1650903498:85 , srv1675101671:85 , srv1825981957:85 , srv1848901335:85 , srv1923345559:85 , srv2018140672:85 , srv2072752412:85 , srv2106106884:85 , srv909396592:85 , srv914607579:85 , srv936747551:85 , srv1178442261:86 , srv1182663925:86 , srv1220106377:86 , srv1259100093:86 , srv1300701442:86 , srv1300824279:86 , srv1327925253:86 , srv1608684249:86 , srv1731164529:86 , srv1732344039:86 , srv1805742701:86 , srv1827864715:86 , srv1839495433:86 , srv1941878633:86 , srv476794657:86 , srv534742078:86 , srv579412003:86 , srv692373354:86 , srv12734738:87 , srv1774341004:87 , srv18544311:87 , srv1862216741:87 , srv1988648884:87 , srv2050854110:87 , srv2120410572:87 , srv885695930:87 , srv991170354:87 , srv1008460674:88 , srv1290883419:88 , srv143861193:88 , srv1450234880:88 , srv1516989518:88 , srv156309816:88 , srv1672230323:88 , srv1856395132:88 , srv205115543:88 , srv2143241326:88 , srv272398677:88 , srv372901877:88 , srv587728494:88 , srv641156857:88 , srv713484207:88 , srv1310948640:89 , srv1539212680:89 , srv1843235609:89 , srv996088749:89 , srv1226285208:90 , srv146933674:90 , srv2054949915:90 , srv2116081636:90 , srv511815527:90 , srv652769146:90 , srv411648640:91 , srv356049607:92 , srv559713020:93 , srv995944291:509 }
2023-03-23 10:56:06,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 10:56:06,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv272398677=69, srv1178442261=6, srv18544311=48, srv772164310=91, srv1839495433=44, srv1521351696=22, srv2114456759=64, srv1049895207=2, srv2072752412=62, srv505932677=75, srv652185072=86, srv1794285146=38, srv1516989518=21, srv1805742701=39, srv1675539738=31, srv1650903498=28, srv1862216741=50, srv995944291=98, srv2106106884=63, srv476794657=74, srv1958490516=54, srv61356507=83, srv1182663925=7, srv143861193=18, srv2050854110=59, srv508186383=76, srv1756724145=35, srv1672230323=29, srv1941878633=53, srv411648640=73, srv587728494=82, srv713484207=89, srv1531321469=23, srv1675101671=30, srv1220106377=8, srv1310948640=15, srv1608684249=27, srv936747551=95, srv885695930=92, srv2143241326=67, srv1059189728=3, srv1988648884=56, srv1830095843=43, srv968589837=96, srv541450037=79, srv1863544153=51, srv621308985=84, srv1856395132=49, srv914607579=94, srv1923345559=52, srv652769146=87, srv1843329058=46, srv12734738=11, srv909396592=93, srv1226285208=9, srv1300701442=13, srv2054949915=61, srv1290883419=12, srv1377904360=17, srv534742078=78, srv146933674=20, srv1450234880=19, srv1731164529=32, srv724971295=90, srv2120410572=66, srv1129301054=5, srv1827864715=42, srv1993950814=57, srv511815527=77, srv1825981957=41, srv205115543=60, srv2116081636=65, srv1969291383=55, srv641156857=85, srv1259100093=10, srv328750537=70, srv1824363472=40, srv1000838094=0, srv559713020=80, srv1534579809=24, srv1732344039=33, srv996088749=99, srv156309816=26, srv1327925253=16, srv356049607=71, srv1734678055=34, srv1008460674=1, srv1783803381=37, srv1063519069=4, srv991170354=97, srv1539212680=25, srv372901877=72, srv1300824279=14, srv579412003=81, srv692373354=88, srv1843235609=45, srv235928878=68, srv2018140672=58, srv1774341004=36, srv1848901335=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 10:56:06,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 10:56:06,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 10:56:06,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 10:56:06,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 10:56:06,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 10:56:06,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 10:56:06,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 10:56:06,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 10:56:06,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 10:56:06,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 10:56:06,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 10:56:06,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 10:56:06,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 10:56:06,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 10:56:06,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 10:56:06,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 10:56:06,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 10:56:06,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 10:56:06,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 10:56:06,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 10:56:06,777 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021600737125608054); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033457680524552995, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021600737125608054); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:06,777 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 10:56:06,777 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01591195208090913, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021600737125608054); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033457680524552995, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021600737125608054); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 10:56:06,995 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 321 ms to try 5000 different iterations.  Found a solution that moves 979 regions; Going from a computed imbalance of 0.01591195208090913 to a new imbalance of 8.89168911813987E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005647941156443356); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00663291117550788); MoveCostFunction : (multiplier=7.0, imbalance=0.10877777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005647941156443356); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:06,997 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1226285208:81 , srv1182663925:82 , srv652185072:82 , srv1178442261:83 , srv1650903498:83 , srv1774341004:83 , srv2054949915:83 , srv1220106377:84 , srv1259100093:84 , srv1300824279:84 , srv1539212680:84 , srv1732344039:84 , srv1827864715:84 , srv1839495433:84 , srv1856395132:84 , srv1941878633:84 , srv1988648884:84 , srv205115543:84 , srv2116081636:84 , srv411648640:84 , srv641156857:84 , srv772164310:84 , srv885695930:84 , srv909396592:84 , srv991170354:84 , srv996088749:84 , srv1049895207:85 , srv1129301054:85 , srv12734738:85 , srv1290883419:85 , srv1310948640:85 , srv1377904360:85 , srv1450234880:85 , srv146933674:85 , srv1516989518:85 , srv1608684249:85 , srv1675101671:85 , srv1675539738:85 , srv1794285146:85 , srv1805742701:85 , srv1824363472:85 , srv1862216741:85 , srv2018140672:85 , srv2050854110:85 , srv2072752412:85 , srv2143241326:85 , srv356049607:85 , srv476794657:85 , srv511815527:85 , srv559713020:85 , srv652769146:85 , srv713484207:85 , srv724971295:85 , srv914607579:85 , srv1000838094:86 , srv1531321469:86 , srv1534579809:86 , srv156309816:86 , srv1783803381:86 , srv1825981957:86 , srv18544311:86 , srv2114456759:86 , srv2120410572:86 , srv328750537:86 , srv505932677:86 , srv508186383:86 , srv534742078:86 , srv541450037:86 , srv61356507:86 , srv621308985:86 , srv936747551:86 , srv1008460674:87 , srv1063519069:87 , srv1327925253:87 , srv143861193:87 , srv1521351696:87 , srv1672230323:87 , srv1731164529:87 , srv1756724145:87 , srv1830095843:87 , srv1843329058:87 , srv1848901335:87 , srv1923345559:87 , srv1958490516:87 , srv1969291383:87 , srv2106106884:87 , srv235928878:87 , srv272398677:87 , srv372901877:87 , srv579412003:87 , srv587728494:87 , srv692373354:87 , srv1059189728:88 , srv1300701442:88 , srv1734678055:88 , srv1843235609:88 , srv1863544153:88 , srv1993950814:88 , srv968589837:88 , srv995944291:542 }
2023-03-23 10:56:07,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 10:56:07,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv272398677=69, srv1178442261=6, srv18544311=48, srv772164310=91, srv1839495433=44, srv1521351696=22, srv2114456759=64, srv1049895207=2, srv2072752412=62, srv505932677=75, srv652185072=86, srv1794285146=38, srv1516989518=21, srv1805742701=39, srv1675539738=31, srv1650903498=28, srv1862216741=50, srv995944291=98, srv2106106884=63, srv476794657=74, srv1958490516=54, srv61356507=83, srv1182663925=7, srv143861193=18, srv2050854110=59, srv508186383=76, srv1756724145=35, srv1672230323=29, srv1941878633=53, srv411648640=73, srv587728494=82, srv713484207=89, srv1531321469=23, srv1675101671=30, srv1220106377=8, srv1310948640=15, srv1608684249=27, srv936747551=95, srv885695930=92, srv2143241326=67, srv1059189728=3, srv1988648884=56, srv1830095843=43, srv968589837=96, srv541450037=79, srv1863544153=51, srv621308985=84, srv1856395132=49, srv914607579=94, srv1923345559=52, srv652769146=87, srv1843329058=46, srv12734738=11, srv909396592=93, srv1226285208=9, srv1300701442=13, srv2054949915=61, srv1290883419=12, srv1377904360=17, srv534742078=78, srv146933674=20, srv1450234880=19, srv1731164529=32, srv724971295=90, srv2120410572=66, srv1129301054=5, srv1827864715=42, srv1993950814=57, srv511815527=77, srv1825981957=41, srv205115543=60, srv2116081636=65, srv1969291383=55, srv641156857=85, srv1259100093=10, srv328750537=70, srv1824363472=40, srv1000838094=0, srv559713020=80, srv1534579809=24, srv1732344039=33, srv996088749=99, srv156309816=26, srv1327925253=16, srv356049607=71, srv1734678055=34, srv1008460674=1, srv1783803381=37, srv1063519069=4, srv991170354=97, srv1539212680=25, srv372901877=72, srv1300824279=14, srv579412003=81, srv692373354=88, srv1843235609=45, srv235928878=68, srv2018140672=58, srv1774341004=36, srv1848901335=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 10:56:07,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 10:56:07,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 10:56:07,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 10:56:07,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 10:56:07,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 10:56:07,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 10:56:07,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 10:56:07,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 10:56:07,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 10:56:07,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 10:56:07,132 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005647941156443356); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00663291117550788); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005647941156443356); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:07,132 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 10:56:07,132 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.823138072969004E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005647941156443356); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00663291117550788); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005647941156443356); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 10:56:07,454 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 419 ms to try 5000 different iterations.  Found a solution that moves 234 regions; Going from a computed imbalance of 8.823138072969004E-4 to a new imbalance of 2.879184006973125E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023210354127426377); MoveCostFunction : (multiplier=7.0, imbalance=0.026, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:07,455 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1327925253:83 , srv1734678055:83 , srv1830095843:83 , srv61356507:83 , srv1008460674:84 , srv1059189728:84 , srv1063519069:84 , srv1226285208:84 , srv1300701442:84 , srv1310948640:84 , srv143861193:84 , srv1450234880:84 , srv1521351696:84 , srv1539212680:84 , srv1672230323:84 , srv1731164529:84 , srv1756724145:84 , srv1824363472:84 , srv1843235609:84 , srv1843329058:84 , srv1848901335:84 , srv1856395132:84 , srv1862216741:84 , srv1863544153:84 , srv1969291383:84 , srv1993950814:84 , srv2018140672:84 , srv2050854110:84 , srv2106106884:84 , srv2114456759:84 , srv235928878:84 , srv328750537:84 , srv356049607:84 , srv372901877:84 , srv411648640:84 , srv508186383:84 , srv511815527:84 , srv579412003:84 , srv652769146:84 , srv713484207:84 , srv724971295:84 , srv909396592:84 , srv914607579:84 , srv936747551:84 , srv996088749:84 , srv1000838094:85 , srv1049895207:85 , srv1129301054:85 , srv1178442261:85 , srv1182663925:85 , srv1220106377:85 , srv1259100093:85 , srv12734738:85 , srv1290883419:85 , srv1300824279:85 , srv1377904360:85 , srv146933674:85 , srv1516989518:85 , srv1531321469:85 , srv1534579809:85 , srv156309816:85 , srv1608684249:85 , srv1650903498:85 , srv1675101671:85 , srv1675539738:85 , srv1732344039:85 , srv1774341004:85 , srv1783803381:85 , srv1794285146:85 , srv1805742701:85 , srv1825981957:85 , srv1827864715:85 , srv1839495433:85 , srv18544311:85 , srv1923345559:85 , srv1941878633:85 , srv1958490516:85 , srv1988648884:85 , srv205115543:85 , srv2054949915:85 , srv2072752412:85 , srv2116081636:85 , srv2120410572:85 , srv2143241326:85 , srv272398677:85 , srv476794657:85 , srv505932677:85 , srv534742078:85 , srv541450037:85 , srv559713020:85 , srv587728494:85 , srv621308985:85 , srv641156857:85 , srv652185072:85 , srv692373354:85 , srv772164310:85 , srv885695930:85 , srv968589837:85 , srv991170354:85 , srv995944291:634 }
2023-03-23 10:56:07,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 10:56:07,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv272398677=69, srv1178442261=6, srv18544311=48, srv772164310=91, srv1839495433=44, srv1521351696=22, srv2114456759=64, srv1049895207=2, srv2072752412=62, srv505932677=75, srv652185072=86, srv1794285146=38, srv1516989518=21, srv1805742701=39, srv1675539738=31, srv1650903498=28, srv1862216741=50, srv995944291=98, srv2106106884=63, srv476794657=74, srv1958490516=54, srv61356507=83, srv1182663925=7, srv143861193=18, srv2050854110=59, srv508186383=76, srv1756724145=35, srv1672230323=29, srv1941878633=53, srv411648640=73, srv587728494=82, srv713484207=89, srv1531321469=23, srv1675101671=30, srv1220106377=8, srv1310948640=15, srv1608684249=27, srv936747551=95, srv885695930=92, srv2143241326=67, srv1059189728=3, srv1988648884=56, srv1830095843=43, srv968589837=96, srv541450037=79, srv1863544153=51, srv621308985=84, srv1856395132=49, srv914607579=94, srv1923345559=52, srv652769146=87, srv1843329058=46, srv12734738=11, srv909396592=93, srv1226285208=9, srv1300701442=13, srv2054949915=61, srv1290883419=12, srv1377904360=17, srv534742078=78, srv146933674=20, srv1450234880=19, srv1731164529=32, srv724971295=90, srv2120410572=66, srv1129301054=5, srv1827864715=42, srv1993950814=57, srv511815527=77, srv1825981957=41, srv205115543=60, srv2116081636=65, srv1969291383=55, srv641156857=85, srv1259100093=10, srv328750537=70, srv1824363472=40, srv1000838094=0, srv559713020=80, srv1534579809=24, srv1732344039=33, srv996088749=99, srv156309816=26, srv1327925253=16, srv356049607=71, srv1734678055=34, srv1008460674=1, srv1783803381=37, srv1063519069=4, srv991170354=97, srv1539212680=25, srv372901877=72, srv1300824279=14, srv579412003=81, srv692373354=88, srv1843235609=45, srv235928878=68, srv2018140672=58, srv1774341004=36, srv1848901335=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 10:56:07,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 10:56:07,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 10:56:07,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 10:56:07,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 10:56:07,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 10:56:07,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 10:56:07,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 10:56:07,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 10:56:07,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 10:56:07,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 10:56:07,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 10:56:07,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 10:56:07,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 10:56:07,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 10:56:07,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 10:56:07,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 10:56:07,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 10:56:07,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 10:56:07,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 10:56:07,602 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023210354127426377); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:07,602 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 10:56:07,603 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.7153337049303976E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023210354127426377); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 10:56:07,805 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 305 ms to try 5000 different iterations.  Found a solution that moves 34 regions; Going from a computed imbalance of 2.7153337049303976E-5 to a new imbalance of 9.031263327756946E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.003777777777777778); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:07,805 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1783803381:83 , srv1923345559:83 , srv1941878633:83 , srv541450037:83 , srv692373354:83 , srv1000838094:84 , srv1008460674:84 , srv1049895207:84 , srv1059189728:84 , srv1129301054:84 , srv1178442261:84 , srv1182663925:84 , srv1220106377:84 , srv1226285208:84 , srv1259100093:84 , srv12734738:84 , srv1290883419:84 , srv1300701442:84 , srv1300824279:84 , srv1310948640:84 , srv1327925253:84 , srv1377904360:84 , srv143861193:84 , srv1450234880:84 , srv146933674:84 , srv1516989518:84 , srv1521351696:84 , srv1531321469:84 , srv1534579809:84 , srv1539212680:84 , srv156309816:84 , srv1608684249:84 , srv1650903498:84 , srv1672230323:84 , srv1675101671:84 , srv1675539738:84 , srv1731164529:84 , srv1732344039:84 , srv1734678055:84 , srv1756724145:84 , srv1774341004:84 , srv1794285146:84 , srv1805742701:84 , srv1824363472:84 , srv1825981957:84 , srv1827864715:84 , srv1830095843:84 , srv1839495433:84 , srv1843235609:84 , srv1848901335:84 , srv18544311:84 , srv1856395132:84 , srv1863544153:84 , srv1958490516:84 , srv1969291383:84 , srv1988648884:84 , srv1993950814:84 , srv2050854110:84 , srv205115543:84 , srv2054949915:84 , srv2072752412:84 , srv2106106884:84 , srv2114456759:84 , srv2116081636:84 , srv2120410572:84 , srv2143241326:84 , srv235928878:84 , srv272398677:84 , srv328750537:84 , srv356049607:84 , srv372901877:84 , srv411648640:84 , srv476794657:84 , srv505932677:84 , srv508186383:84 , srv511815527:84 , srv534742078:84 , srv559713020:84 , srv579412003:84 , srv587728494:84 , srv61356507:84 , srv621308985:84 , srv641156857:84 , srv652185072:84 , srv652769146:84 , srv713484207:84 , srv724971295:84 , srv772164310:84 , srv885695930:84 , srv909396592:84 , srv936747551:84 , srv968589837:84 , srv991170354:84 , srv996088749:84 , srv1063519069:85 , srv1843329058:85 , srv1862216741:85 , srv2018140672:85 , srv914607579:85 , srv995944291:684 }
2023-03-23 10:56:07,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 10:56:07,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv272398677=69, srv1178442261=6, srv18544311=48, srv772164310=91, srv1839495433=44, srv1521351696=22, srv2114456759=64, srv1049895207=2, srv2072752412=62, srv505932677=75, srv652185072=86, srv1794285146=38, srv1516989518=21, srv1805742701=39, srv1675539738=31, srv1650903498=28, srv1862216741=50, srv995944291=98, srv2106106884=63, srv476794657=74, srv1958490516=54, srv61356507=83, srv1182663925=7, srv143861193=18, srv2050854110=59, srv508186383=76, srv1756724145=35, srv1672230323=29, srv1941878633=53, srv411648640=73, srv587728494=82, srv713484207=89, srv1531321469=23, srv1675101671=30, srv1220106377=8, srv1310948640=15, srv1608684249=27, srv936747551=95, srv885695930=92, srv2143241326=67, srv1059189728=3, srv1988648884=56, srv1830095843=43, srv968589837=96, srv541450037=79, srv1863544153=51, srv621308985=84, srv1856395132=49, srv914607579=94, srv1923345559=52, srv652769146=87, srv1843329058=46, srv12734738=11, srv909396592=93, srv1226285208=9, srv1300701442=13, srv2054949915=61, srv1290883419=12, srv1377904360=17, srv534742078=78, srv146933674=20, srv1450234880=19, srv1731164529=32, srv724971295=90, srv2120410572=66, srv1129301054=5, srv1827864715=42, srv1993950814=57, srv511815527=77, srv1825981957=41, srv205115543=60, srv2116081636=65, srv1969291383=55, srv641156857=85, srv1259100093=10, srv328750537=70, srv1824363472=40, srv1000838094=0, srv559713020=80, srv1534579809=24, srv1732344039=33, srv996088749=99, srv156309816=26, srv1327925253=16, srv356049607=71, srv1734678055=34, srv1008460674=1, srv1783803381=37, srv1063519069=4, srv991170354=97, srv1539212680=25, srv372901877=72, srv1300824279=14, srv579412003=81, srv692373354=88, srv1843235609=45, srv235928878=68, srv2018140672=58, srv1774341004=36, srv1848901335=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 10:56:07,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 10:56:07,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 10:56:07,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 10:56:07,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 10:56:07,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 10:56:07,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 10:56:07,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 10:56:07,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 10:56:07,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 10:56:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 10:56:07,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 10:56:07,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 10:56:07,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 10:56:07,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 10:56:07,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 10:56:07,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 10:56:07,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 10:56:07,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 10:56:07,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 10:56:07,935 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.793190239318795E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:07,936 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1783803381:83 , srv1923345559:83 , srv1941878633:83 , srv541450037:83 , srv692373354:83 , srv1000838094:84 , srv1008460674:84 , srv1049895207:84 , srv1059189728:84 , srv1129301054:84 , srv1178442261:84 , srv1182663925:84 , srv1220106377:84 , srv1226285208:84 , srv1259100093:84 , srv12734738:84 , srv1290883419:84 , srv1300701442:84 , srv1300824279:84 , srv1310948640:84 , srv1327925253:84 , srv1377904360:84 , srv143861193:84 , srv1450234880:84 , srv146933674:84 , srv1516989518:84 , srv1521351696:84 , srv1531321469:84 , srv1534579809:84 , srv1539212680:84 , srv156309816:84 , srv1608684249:84 , srv1650903498:84 , srv1672230323:84 , srv1675101671:84 , srv1675539738:84 , srv1731164529:84 , srv1732344039:84 , srv1734678055:84 , srv1756724145:84 , srv1774341004:84 , srv1794285146:84 , srv1805742701:84 , srv1824363472:84 , srv1825981957:84 , srv1827864715:84 , srv1830095843:84 , srv1839495433:84 , srv1843235609:84 , srv1848901335:84 , srv18544311:84 , srv1856395132:84 , srv1863544153:84 , srv1958490516:84 , srv1969291383:84 , srv1988648884:84 , srv1993950814:84 , srv2050854110:84 , srv205115543:84 , srv2054949915:84 , srv2072752412:84 , srv2106106884:84 , srv2114456759:84 , srv2116081636:84 , srv2120410572:84 , srv2143241326:84 , srv235928878:84 , srv272398677:84 , srv328750537:84 , srv356049607:84 , srv372901877:84 , srv411648640:84 , srv476794657:84 , srv505932677:84 , srv508186383:84 , srv511815527:84 , srv534742078:84 , srv559713020:84 , srv579412003:84 , srv587728494:84 , srv61356507:84 , srv621308985:84 , srv641156857:84 , srv652185072:84 , srv652769146:84 , srv713484207:84 , srv724971295:84 , srv772164310:84 , srv885695930:84 , srv909396592:84 , srv936747551:84 , srv968589837:84 , srv991170354:84 , srv996088749:84 , srv1063519069:85 , srv1843329058:85 , srv1862216741:85 , srv2018140672:85 , srv914607579:85 , srv995944291:684 }
2023-03-23 10:56:07,961 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=355 (was 355), ProcessCount=169 (was 169), AvailableMemoryMB=6031 (was 6523)
2023-03-23 10:56:07,969 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=355, ProcessCount=169, AvailableMemoryMB=6031
2023-03-23 10:56:07,970 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-23 10:56:07,970 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-23 10:56:07,978 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1038770613:84 , srv1053881241:84 , srv1123233027:84 , srv1147500878:84 , srv138447076:84 , srv1416928215:84 , srv151755905:84 , srv1565559304:84 , srv1636765257:84 , srv1762048267:84 , srv1972090133:84 , srv2005809697:84 , srv2005935042:84 , srv2057173448:84 , srv2092356595:84 , srv2107757013:84 , srv331230122:84 , srv423091395:84 , srv499108110:84 , srv526279887:84 , srv607900196:84 , srv6788694:84 , srv737457200:84 , srv771649192:84 , srv776693483:84 , srv825085829:84 , srv8668265:84 , srv97199969:84 , srv97900324:84 , srv743265995:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-23 10:56:07,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 10:56:07,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2057173448=13, srv1972090133=10, srv423091395=17, srv6788694=21, srv607900196=20, srv8668265=27, srv743265995=23, srv97199969=28, srv499108110=18, srv1416928215=5, srv776693483=25, srv151755905=6, srv1123233027=2, srv1565559304=7, srv1762048267=9, srv1636765257=8, srv825085829=26, srv97900324=29, srv737457200=22, srv771649192=24, srv526279887=19, srv2005809697=11, srv1053881241=1, srv2005935042=12, srv138447076=4, srv1038770613=0, srv1147500878=3, srv2107757013=15, srv331230122=16, srv2092356595=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 10:56:07,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 10:56:07,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 10:56:07,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 10:56:07,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 10:56:07,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 10:56:07,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 10:56:07,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 10:56:07,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 10:56:07,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 10:56:07,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 10:56:07,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 10:56:07,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-23 10:56:08,004 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1431801552014482, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:56:08,005 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912644199258056, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1431801552014482, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-23 10:58:07,991 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6624598 different iterations.  Found a solution that moves 1853 regions; Going from a computed imbalance of 0.9912644199258056 to a new imbalance of 4.324994439959734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6862962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:58:07,997 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1038770613:90 , srv1053881241:90 , srv1123233027:90 , srv1147500878:90 , srv138447076:90 , srv1416928215:90 , srv151755905:90 , srv1565559304:90 , srv1636765257:90 , srv1762048267:90 , srv1972090133:90 , srv2005809697:90 , srv2005935042:90 , srv2057173448:90 , srv2092356595:90 , srv2107757013:90 , srv331230122:90 , srv423091395:90 , srv499108110:90 , srv526279887:90 , srv607900196:90 , srv6788694:90 , srv737457200:90 , srv743265995:90 , srv771649192:90 , srv776693483:90 , srv825085829:90 , srv8668265:90 , srv97199969:90 , srv97900324:90 }
2023-03-23 10:58:07,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 10:58:07,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2057173448=13, srv1972090133=10, srv423091395=17, srv6788694=21, srv607900196=20, srv8668265=27, srv743265995=23, srv97199969=28, srv499108110=18, srv1416928215=5, srv776693483=25, srv151755905=6, srv1123233027=2, srv1565559304=7, srv1762048267=9, srv1636765257=8, srv825085829=26, srv97900324=29, srv737457200=22, srv771649192=24, srv526279887=19, srv2005809697=11, srv1053881241=1, srv2005935042=12, srv138447076=4, srv1038770613=0, srv1147500878=3, srv2107757013=15, srv331230122=16, srv2092356595=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 10:58:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 10:58:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 10:58:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 10:58:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 10:58:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 10:58:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 10:58:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 10:58:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 10:58:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 10:58:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 10:58:08,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-23 10:58:08,017 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 10:58:08,017 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1038770613:90 , srv1053881241:90 , srv1123233027:90 , srv1147500878:90 , srv138447076:90 , srv1416928215:90 , srv151755905:90 , srv1565559304:90 , srv1636765257:90 , srv1762048267:90 , srv1972090133:90 , srv2005809697:90 , srv2005935042:90 , srv2057173448:90 , srv2092356595:90 , srv2107757013:90 , srv331230122:90 , srv423091395:90 , srv499108110:90 , srv526279887:90 , srv607900196:90 , srv6788694:90 , srv737457200:90 , srv743265995:90 , srv771649192:90 , srv776693483:90 , srv825085829:90 , srv8668265:90 , srv97199969:90 , srv97900324:90 }
2023-03-23 10:58:08,030 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=200 (was 355), ProcessCount=172 (was 169) - ProcessCount LEAK? -, AvailableMemoryMB=6078 (was 6031) - AvailableMemoryMB LEAK? -