2023-03-14 23:54:17,974 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-14 23:54:18,283 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-14 23:54:18,386 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-14 23:54:18,440 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-14 23:54:18,447 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-14 23:54:18,462 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=259, ProcessCount=169, AvailableMemoryMB=3846
2023-03-14 23:54:18,464 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-14 23:54:18,464 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-14 23:54:18,578 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1008083270:84 , srv1012760030:84 , srv1027781054:84 , srv1076127282:84 , srv1101965315:84 , srv111474344:84 , srv1114938577:84 , srv1172935384:84 , srv1200312285:84 , srv1247700779:84 , srv1285448287:84 , srv1297128098:84 , srv1309239573:84 , srv1317949176:84 , srv1385642381:84 , srv1416600423:84 , srv142671848:84 , srv1432839964:84 , srv1444432982:84 , srv1449133167:84 , srv1449743557:84 , srv1462540522:84 , srv1480388955:84 , srv1503303879:84 , srv1510606660:84 , srv1561975773:84 , srv1563609891:84 , srv1571597471:84 , srv1575581340:84 , srv1577747257:84 , srv1579804178:84 , srv1610459948:84 , srv1625120818:84 , srv1635616341:84 , srv1636139619:84 , srv166876104:84 , srv1670224439:84 , srv1673251301:84 , srv1677852614:84 , srv1719822642:84 , srv1723836827:84 , srv1743253187:84 , srv1756560127:84 , srv1762297899:84 , srv1778694112:84 , srv1787837392:84 , srv1792466679:84 , srv1811485332:84 , srv1823106232:84 , srv1838749328:84 , srv18404025:84 , srv1843738090:84 , srv1876082747:84 , srv1884426529:84 , srv1887453811:84 , srv1912262528:84 , srv1916066144:84 , srv1921550511:84 , srv1951496653:84 , srv1970739100:84 , srv1972106253:84 , srv1979391152:84 , srv1980611473:84 , srv2000072077:84 , srv2009215686:84 , srv2059477250:84 , srv2068360164:84 , srv2085013793:84 , srv2090798322:84 , srv2090891823:84 , srv2092411872:84 , srv214018454:84 , srv236153371:84 , srv267116990:84 , srv272029233:84 , srv294997428:84 , srv314095723:84 , srv326885996:84 , srv342849107:84 , srv369430324:84 , srv397392210:84 , srv400543892:84 , srv441785324:84 , srv444894315:84 , srv451842805:84 , srv479795983:84 , srv529575854:84 , srv531262236:84 , srv548868610:84 , srv568013832:84 , srv583465656:84 , srv591932855:84 , srv647332200:84 , srv681323024:84 , srv684008526:84 , srv812952582:84 , srv838636639:84 , srv945199208:84 , srv956556175:84 , srv19206827:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-14 23:54:18,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 23:54:18,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1811485332=47, srv1076127282=3, srv1823106232=48, srv1610459948=31, srv1462540522=21, srv272029233=75, srv1951496653=59, srv1670224439=36, srv647332200=93, srv166876104=35, srv1577747257=29, srv945199208=98, srv1916066144=56, srv451842805=85, srv531262236=88, srv1884426529=53, srv1444432982=18, srv479795983=86, srv2092411872=71, srv956556175=99, srv1635616341=33, srv1449743557=20, srv1743253187=41, srv1172935384=7, srv1008083270=0, srv529575854=87, srv1012760030=1, srv1838749328=49, srv1449133167=19, srv326885996=78, srv1297128098=11, srv1317949176=13, srv1843738090=51, srv1563609891=26, srv400543892=82, srv1385642381=14, srv1579804178=30, srv548868610=89, srv1778694112=44, srv568013832=90, srv2090798322=69, srv1101965315=4, srv1416600423=15, srv1480388955=22, srv1561975773=25, srv2000072077=64, srv1114938577=6, srv1677852614=38, srv342849107=79, srv267116990=74, srv1571597471=27, srv1432839964=17, srv1636139619=34, srv214018454=72, srv236153371=73, srv1979391152=62, srv1503303879=23, srv1887453811=54, srv397392210=81, srv1027781054=2, srv684008526=95, srv1247700779=9, srv1762297899=43, srv19206827=57, srv1970739100=60, srv1309239573=12, srv1972106253=61, srv1723836827=40, srv838636639=97, srv2068360164=67, srv2059477250=66, srv2085013793=68, srv369430324=80, srv441785324=83, srv1719822642=39, srv1510606660=24, srv444894315=84, srv1756560127=42, srv681323024=94, srv1575581340=28, srv294997428=76, srv591932855=92, srv1921550511=58, srv314095723=77, srv583465656=91, srv2090891823=70, srv142671848=16, srv812952582=96, srv1625120818=32, srv1792466679=46, srv1787837392=45, srv1980611473=63, srv111474344=5, srv1876082747=52, srv1285448287=10, srv1673251301=37, srv1912262528=55, srv1200312285=8, srv18404025=50, srv2009215686=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 23:54:18,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 23:54:18,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 23:54:18,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 23:54:18,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 23:54:18,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 23:54:18,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 23:54:18,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 23:54:18,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 23:54:18,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 23:54:18,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 23:54:18,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 23:54:18,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 23:54:18,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 23:54:18,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 23:54:18,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 23:54:18,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 23:54:18,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 23:54:18,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 23:54:18,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 23:54:18,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 23:54:18,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 23:54:18,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 23:54:18,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 23:54:18,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 23:54:18,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 23:54:18,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 23:54:18,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 23:54:18,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 23:54:18,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 23:54:18,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 23:54:18,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-14 23:54:18,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-14 23:54:18,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-14 23:54:18,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-14 23:54:18,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-14 23:54:18,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-14 23:54:18,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-14 23:54:18,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-14 23:54:18,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-14 23:54:18,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-14 23:54:18,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-14 23:54:18,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-14 23:54:18,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-14 23:54:18,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-14 23:54:18,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-14 23:54:18,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-14 23:54:18,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-14 23:54:18,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-14 23:54:18,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-14 23:54:18,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-14 23:54:18,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-14 23:54:18,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-14 23:54:18,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-14 23:54:18,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-14 23:54:18,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-14 23:54:18,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-14 23:54:18,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-14 23:54:18,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-14 23:54:18,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-14 23:54:18,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-14 23:54:18,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-14 23:54:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-14 23:54:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-14 23:54:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-14 23:54:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-14 23:54:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-14 23:54:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-14 23:54:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-14 23:54:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-14 23:54:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-14 23:54:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-14 23:54:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-14 23:54:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-14 23:54:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-14 23:54:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-14 23:54:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-14 23:54:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-14 23:54:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-14 23:54:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-14 23:54:18,856 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:54:18,856 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-14 23:54:18,857 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-14 23:54:19,109 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 382 ms to try 5000 different iterations.  Found a solution that moves 4683 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17801347262118702. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03826646357703577, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05484188065975255, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5203333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03826646357703577, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17341666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.238, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:54:19,119 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv326885996:81 , srv1101965315:82 , srv1114938577:82 , srv1723836827:82 , srv1778694112:82 , srv2059477250:82 , srv838636639:82 , srv1200312285:83 , srv1610459948:83 , srv1635616341:83 , srv2092411872:83 , srv267116990:83 , srv294997428:83 , srv444894315:83 , srv591932855:83 , srv1636139619:84 , srv1912262528:84 , srv2068360164:84 , srv342849107:84 , srv369430324:84 , srv684008526:84 , srv1297128098:85 , srv1385642381:85 , srv1503303879:85 , srv1510606660:85 , srv1563609891:85 , srv1577747257:85 , srv1756560127:85 , srv1787837392:85 , srv272029233:85 , srv441785324:85 , srv451842805:85 , srv1076127282:86 , srv1309239573:86 , srv1444432982:86 , srv1449133167:86 , srv1743253187:86 , srv1811485332:86 , srv1887453811:86 , srv1916066144:86 , srv214018454:86 , srv397392210:86 , srv529575854:86 , srv812952582:86 , srv111474344:87 , srv1449743557:87 , srv1625120818:87 , srv166876104:87 , srv1673251301:87 , srv1677852614:87 , srv1719822642:87 , srv1823106232:87 , srv1838749328:87 , srv1843738090:87 , srv1876082747:87 , srv1951496653:87 , srv1970739100:87 , srv2000072077:87 , srv314095723:87 , srv400543892:87 , srv531262236:87 , srv945199208:87 , srv1285448287:88 , srv1317949176:88 , srv1432839964:88 , srv1480388955:88 , srv1792466679:88 , srv18404025:88 , srv1980611473:88 , srv2085013793:88 , srv2090798322:88 , srv2090891823:88 , srv236153371:88 , srv479795983:88 , srv548868610:88 , srv568013832:88 , srv583465656:88 , srv647332200:88 , srv956556175:88 , srv1008083270:89 , srv1027781054:89 , srv1172935384:89 , srv1416600423:89 , srv142671848:89 , srv1571597471:89 , srv1579804178:89 , srv1921550511:89 , srv1972106253:89 , srv1979391152:89 , srv681323024:89 , srv1247700779:90 , srv1575581340:90 , srv1670224439:90 , srv1762297899:90 , srv1884426529:90 , srv1012760030:91 , srv1561975773:91 , srv1462540522:94 , srv2009215686:95 , srv19206827:430 }
2023-03-14 23:54:19,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 23:54:19,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1811485332=47, srv1076127282=3, srv1823106232=48, srv1610459948=31, srv1462540522=21, srv272029233=75, srv1951496653=59, srv1670224439=36, srv647332200=93, srv166876104=35, srv1577747257=29, srv945199208=98, srv1916066144=56, srv451842805=85, srv531262236=88, srv1884426529=53, srv1444432982=18, srv479795983=86, srv2092411872=71, srv956556175=99, srv1635616341=33, srv1449743557=20, srv1743253187=41, srv1172935384=7, srv1008083270=0, srv529575854=87, srv1012760030=1, srv1838749328=49, srv1449133167=19, srv326885996=78, srv1297128098=11, srv1317949176=13, srv1843738090=51, srv1563609891=26, srv400543892=82, srv1385642381=14, srv1579804178=30, srv548868610=89, srv1778694112=44, srv568013832=90, srv2090798322=69, srv1101965315=4, srv1416600423=15, srv1480388955=22, srv1561975773=25, srv2000072077=64, srv1114938577=6, srv1677852614=38, srv342849107=79, srv267116990=74, srv1571597471=27, srv1432839964=17, srv1636139619=34, srv214018454=72, srv236153371=73, srv1979391152=62, srv1503303879=23, srv1887453811=54, srv397392210=81, srv1027781054=2, srv684008526=95, srv1247700779=9, srv1762297899=43, srv19206827=57, srv1970739100=60, srv1309239573=12, srv1972106253=61, srv1723836827=40, srv838636639=97, srv2068360164=67, srv2059477250=66, srv2085013793=68, srv369430324=80, srv441785324=83, srv1719822642=39, srv1510606660=24, srv444894315=84, srv1756560127=42, srv681323024=94, srv1575581340=28, srv294997428=76, srv591932855=92, srv1921550511=58, srv314095723=77, srv583465656=91, srv2090891823=70, srv142671848=16, srv812952582=96, srv1625120818=32, srv1792466679=46, srv1787837392=45, srv1980611473=63, srv111474344=5, srv1876082747=52, srv1285448287=10, srv1673251301=37, srv1912262528=55, srv1200312285=8, srv18404025=50, srv2009215686=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 23:54:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 23:54:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 23:54:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 23:54:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 23:54:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 23:54:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 23:54:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 23:54:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 23:54:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 23:54:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 23:54:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 23:54:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 23:54:19,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-14 23:54:19,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-14 23:54:19,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-14 23:54:19,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-14 23:54:19,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-14 23:54:19,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-14 23:54:19,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-14 23:54:19,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-14 23:54:19,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-14 23:54:19,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-14 23:54:19,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-14 23:54:19,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-14 23:54:19,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-14 23:54:19,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-14 23:54:19,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-14 23:54:19,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-14 23:54:19,267 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03826646357703577, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05484188065975255, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03826646357703577, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17341666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.238, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:54:19,268 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-14 23:54:19,268 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1779806815543295, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03826646357703577, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05484188065975255, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03826646357703577, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17341666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.238, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-14 23:54:19,542 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 373 ms to try 5000 different iterations.  Found a solution that moves 2703 regions; Going from a computed imbalance of 0.1779806815543295 to a new imbalance of 0.01476737010840075. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020716074799094566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029259472359962155, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.30033333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020716074799094566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044583333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:54:19,551 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2009215686:77 , srv1561975773:78 , srv1884426529:80 , srv1008083270:81 , srv1462540522:81 , srv1670224439:81 , srv1979391152:81 , srv1012760030:82 , srv1247700779:82 , srv1416600423:82 , srv142671848:82 , srv1762297899:82 , srv2090891823:82 , srv1027781054:83 , srv1571597471:83 , srv1579804178:83 , srv1876082747:83 , srv1972106253:83 , srv2090798322:83 , srv400543892:83 , srv479795983:83 , srv548868610:83 , srv111474344:84 , srv1317949176:84 , srv1563609891:84 , srv1575581340:84 , srv1673251301:84 , srv1823106232:84 , srv1838749328:84 , srv1843738090:84 , srv1921550511:84 , srv314095723:84 , srv441785324:84 , srv568013832:84 , srv945199208:84 , srv1449133167:85 , srv1449743557:85 , srv1677852614:85 , srv1719822642:85 , srv1787837392:85 , srv1792466679:85 , srv18404025:85 , srv1970739100:85 , srv1980611473:85 , srv583465656:85 , srv681323024:85 , srv956556175:85 , srv1172935384:86 , srv1285448287:86 , srv1309239573:86 , srv1432839964:86 , srv1480388955:86 , srv1577747257:86 , srv1625120818:86 , srv1635616341:86 , srv166876104:86 , srv1743253187:86 , srv2000072077:86 , srv531262236:86 , srv647332200:86 , srv1076127282:87 , srv1444432982:87 , srv1503303879:87 , srv1510606660:87 , srv1811485332:87 , srv1887453811:87 , srv1912262528:87 , srv1916066144:87 , srv1951496653:87 , srv2085013793:87 , srv214018454:87 , srv236153371:87 , srv294997428:87 , srv342849107:87 , srv369430324:87 , srv397392210:87 , srv529575854:87 , srv684008526:87 , srv1297128098:88 , srv1385642381:88 , srv1610459948:88 , srv1636139619:88 , srv1756560127:88 , srv272029233:88 , srv451842805:88 , srv838636639:88 , srv1200312285:89 , srv1723836827:89 , srv2068360164:89 , srv444894315:89 , srv812952582:89 , srv1114938577:90 , srv1778694112:90 , srv2059477250:90 , srv591932855:90 , srv2092411872:91 , srv267116990:92 , srv1101965315:93 , srv326885996:93 , srv19206827:528 }
2023-03-14 23:54:19,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 23:54:19,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1811485332=47, srv1076127282=3, srv1823106232=48, srv1610459948=31, srv1462540522=21, srv272029233=75, srv1951496653=59, srv1670224439=36, srv647332200=93, srv166876104=35, srv1577747257=29, srv945199208=98, srv1916066144=56, srv451842805=85, srv531262236=88, srv1884426529=53, srv1444432982=18, srv479795983=86, srv2092411872=71, srv956556175=99, srv1635616341=33, srv1449743557=20, srv1743253187=41, srv1172935384=7, srv1008083270=0, srv529575854=87, srv1012760030=1, srv1838749328=49, srv1449133167=19, srv326885996=78, srv1297128098=11, srv1317949176=13, srv1843738090=51, srv1563609891=26, srv400543892=82, srv1385642381=14, srv1579804178=30, srv548868610=89, srv1778694112=44, srv568013832=90, srv2090798322=69, srv1101965315=4, srv1416600423=15, srv1480388955=22, srv1561975773=25, srv2000072077=64, srv1114938577=6, srv1677852614=38, srv342849107=79, srv267116990=74, srv1571597471=27, srv1432839964=17, srv1636139619=34, srv214018454=72, srv236153371=73, srv1979391152=62, srv1503303879=23, srv1887453811=54, srv397392210=81, srv1027781054=2, srv684008526=95, srv1247700779=9, srv1762297899=43, srv19206827=57, srv1970739100=60, srv1309239573=12, srv1972106253=61, srv1723836827=40, srv838636639=97, srv2068360164=67, srv2059477250=66, srv2085013793=68, srv369430324=80, srv441785324=83, srv1719822642=39, srv1510606660=24, srv444894315=84, srv1756560127=42, srv681323024=94, srv1575581340=28, srv294997428=76, srv591932855=92, srv1921550511=58, srv314095723=77, srv583465656=91, srv2090891823=70, srv142671848=16, srv812952582=96, srv1625120818=32, srv1792466679=46, srv1787837392=45, srv1980611473=63, srv111474344=5, srv1876082747=52, srv1285448287=10, srv1673251301=37, srv1912262528=55, srv1200312285=8, srv18404025=50, srv2009215686=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 23:54:19,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 23:54:19,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 23:54:19,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 23:54:19,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 23:54:19,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 23:54:19,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-14 23:54:19,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-14 23:54:19,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-14 23:54:19,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-14 23:54:19,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-14 23:54:19,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-14 23:54:19,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-14 23:54:19,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-14 23:54:19,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-14 23:54:19,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-14 23:54:19,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-14 23:54:19,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-14 23:54:19,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-14 23:54:19,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-14 23:54:19,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-14 23:54:19,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-14 23:54:19,687 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020716074799094566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029259472359962155, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020716074799094566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044583333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:54:19,687 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-14 23:54:19,687 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014748443297869918, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020716074799094566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029259472359962155, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020716074799094566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044583333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-14 23:54:19,966 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 378 ms to try 5000 different iterations.  Found a solution that moves 955 regions; Going from a computed imbalance of 0.014748443297869918 to a new imbalance of 3.740490644081214E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004877857158646318); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006391633139214696); MoveCostFunction : (multiplier=7.0, imbalance=0.10611111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004877857158646318); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:54:19,970 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv267116990:81 , srv1462540522:82 , srv1101965315:83 , srv1432839964:83 , srv1444432982:83 , srv236153371:83 , srv326885996:83 , srv647332200:83 , srv681323024:83 , srv812952582:83 , srv1012760030:84 , srv1172935384:84 , srv1285448287:84 , srv1480388955:84 , srv1575581340:84 , srv1756560127:84 , srv1792466679:84 , srv1887453811:84 , srv1916066144:84 , srv1921550511:84 , srv1951496653:84 , srv1980611473:84 , srv2000072077:84 , srv2068360164:84 , srv2085013793:84 , srv2092411872:84 , srv214018454:84 , srv272029233:84 , srv529575854:84 , srv1027781054:85 , srv1076127282:85 , srv1114938577:85 , srv1247700779:85 , srv1297128098:85 , srv1385642381:85 , srv1510606660:85 , srv1579804178:85 , srv1625120818:85 , srv166876104:85 , srv1762297899:85 , srv1811485332:85 , srv18404025:85 , srv1972106253:85 , srv2009215686:85 , srv2059477250:85 , srv397392210:85 , srv451842805:85 , srv531262236:85 , srv591932855:85 , srv956556175:85 , srv1200312285:86 , srv1309239573:86 , srv1317949176:86 , srv1416600423:86 , srv142671848:86 , srv1449133167:86 , srv1449743557:86 , srv1571597471:86 , srv1577747257:86 , srv1636139619:86 , srv1677852614:86 , srv1719822642:86 , srv1743253187:86 , srv1778694112:86 , srv1838749328:86 , srv1843738090:86 , srv1912262528:86 , srv1970739100:86 , srv2090798322:86 , srv314095723:86 , srv444894315:86 , srv479795983:86 , srv568013832:86 , srv583465656:86 , srv945199208:86 , srv111474344:87 , srv1503303879:87 , srv1610459948:87 , srv1673251301:87 , srv1723836827:87 , srv1787837392:87 , srv1823106232:87 , srv1884426529:87 , srv1979391152:87 , srv342849107:87 , srv400543892:87 , srv548868610:87 , srv684008526:87 , srv838636639:87 , srv1008083270:88 , srv1561975773:88 , srv1563609891:88 , srv1635616341:88 , srv1670224439:88 , srv1876082747:88 , srv2090891823:88 , srv294997428:88 , srv369430324:88 , srv441785324:89 , srv19206827:543 }
2023-03-14 23:54:19,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 23:54:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1811485332=47, srv1076127282=3, srv1823106232=48, srv1610459948=31, srv1462540522=21, srv272029233=75, srv1951496653=59, srv1670224439=36, srv647332200=93, srv166876104=35, srv1577747257=29, srv945199208=98, srv1916066144=56, srv451842805=85, srv531262236=88, srv1884426529=53, srv1444432982=18, srv479795983=86, srv2092411872=71, srv956556175=99, srv1635616341=33, srv1449743557=20, srv1743253187=41, srv1172935384=7, srv1008083270=0, srv529575854=87, srv1012760030=1, srv1838749328=49, srv1449133167=19, srv326885996=78, srv1297128098=11, srv1317949176=13, srv1843738090=51, srv1563609891=26, srv400543892=82, srv1385642381=14, srv1579804178=30, srv548868610=89, srv1778694112=44, srv568013832=90, srv2090798322=69, srv1101965315=4, srv1416600423=15, srv1480388955=22, srv1561975773=25, srv2000072077=64, srv1114938577=6, srv1677852614=38, srv342849107=79, srv267116990=74, srv1571597471=27, srv1432839964=17, srv1636139619=34, srv214018454=72, srv236153371=73, srv1979391152=62, srv1503303879=23, srv1887453811=54, srv397392210=81, srv1027781054=2, srv684008526=95, srv1247700779=9, srv1762297899=43, srv19206827=57, srv1970739100=60, srv1309239573=12, srv1972106253=61, srv1723836827=40, srv838636639=97, srv2068360164=67, srv2059477250=66, srv2085013793=68, srv369430324=80, srv441785324=83, srv1719822642=39, srv1510606660=24, srv444894315=84, srv1756560127=42, srv681323024=94, srv1575581340=28, srv294997428=76, srv591932855=92, srv1921550511=58, srv314095723=77, srv583465656=91, srv2090891823=70, srv142671848=16, srv812952582=96, srv1625120818=32, srv1792466679=46, srv1787837392=45, srv1980611473=63, srv111474344=5, srv1876082747=52, srv1285448287=10, srv1673251301=37, srv1912262528=55, srv1200312285=8, srv18404025=50, srv2009215686=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 23:54:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 23:54:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 23:54:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 23:54:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 23:54:19,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 23:54:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 23:54:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 23:54:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 23:54:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 23:54:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 23:54:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 23:54:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 23:54:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 23:54:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 23:54:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 23:54:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 23:54:19,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-14 23:54:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-14 23:54:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-14 23:54:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-14 23:54:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-14 23:54:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-14 23:54:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-14 23:54:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-14 23:54:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-14 23:54:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-14 23:54:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-14 23:54:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-14 23:54:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-14 23:54:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-14 23:54:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-14 23:54:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-14 23:54:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-14 23:54:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-14 23:54:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-14 23:54:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-14 23:54:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-14 23:54:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-14 23:54:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-14 23:54:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-14 23:54:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-14 23:54:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-14 23:54:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-14 23:54:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-14 23:54:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-14 23:54:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-14 23:54:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-14 23:54:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-14 23:54:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-14 23:54:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-14 23:54:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-14 23:54:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-14 23:54:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-14 23:54:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-14 23:54:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-14 23:54:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-14 23:54:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-14 23:54:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-14 23:54:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-14 23:54:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-14 23:54:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-14 23:54:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-14 23:54:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-14 23:54:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-14 23:54:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-14 23:54:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-14 23:54:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-14 23:54:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-14 23:54:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-14 23:54:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-14 23:54:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-14 23:54:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-14 23:54:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-14 23:54:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-14 23:54:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-14 23:54:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-14 23:54:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-14 23:54:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-14 23:54:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-14 23:54:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-14 23:54:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-14 23:54:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-14 23:54:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-14 23:54:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-14 23:54:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-14 23:54:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-14 23:54:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-14 23:54:20,123 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004877857158646318); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006391633139214696); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004877857158646318); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:54:20,124 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-14 23:54:20,124 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.673620114828734E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004877857158646318); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006391633139214696); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004877857158646318); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-14 23:54:20,406 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 381 ms to try 5000 different iterations.  Found a solution that moves 233 regions; Going from a computed imbalance of 3.673620114828734E-4 to a new imbalance of 1.5118521750436226E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024618298195866546); MoveCostFunction : (multiplier=7.0, imbalance=0.02588888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:54:20,407 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1172935384:83 , srv1970739100:83 , srv2085013793:83 , srv369430324:83 , srv441785324:83 , srv1076127282:84 , srv111474344:84 , srv1200312285:84 , srv1285448287:84 , srv1309239573:84 , srv1317949176:84 , srv1385642381:84 , srv1432839964:84 , srv1449743557:84 , srv1480388955:84 , srv1503303879:84 , srv1563609891:84 , srv1577747257:84 , srv1610459948:84 , srv1625120818:84 , srv1635616341:84 , srv1636139619:84 , srv166876104:84 , srv1670224439:84 , srv1673251301:84 , srv1677852614:84 , srv1719822642:84 , srv1723836827:84 , srv1743253187:84 , srv1778694112:84 , srv1787837392:84 , srv1792466679:84 , srv1811485332:84 , srv1823106232:84 , srv18404025:84 , srv1876082747:84 , srv1884426529:84 , srv1887453811:84 , srv1951496653:84 , srv2090891823:84 , srv2092411872:84 , srv236153371:84 , srv294997428:84 , srv342849107:84 , srv397392210:84 , srv444894315:84 , srv451842805:84 , srv531262236:84 , srv548868610:84 , srv568013832:84 , srv583465656:84 , srv591932855:84 , srv684008526:84 , srv812952582:84 , srv956556175:84 , srv1008083270:85 , srv1012760030:85 , srv1027781054:85 , srv1101965315:85 , srv1114938577:85 , srv1247700779:85 , srv1297128098:85 , srv1416600423:85 , srv142671848:85 , srv1449133167:85 , srv1462540522:85 , srv1510606660:85 , srv1561975773:85 , srv1571597471:85 , srv1575581340:85 , srv1579804178:85 , srv1756560127:85 , srv1762297899:85 , srv1838749328:85 , srv1843738090:85 , srv1912262528:85 , srv1916066144:85 , srv1921550511:85 , srv1972106253:85 , srv2000072077:85 , srv2009215686:85 , srv2059477250:85 , srv2068360164:85 , srv2090798322:85 , srv214018454:85 , srv272029233:85 , srv314095723:85 , srv326885996:85 , srv400543892:85 , srv479795983:85 , srv529575854:85 , srv647332200:85 , srv681323024:85 , srv838636639:85 , srv945199208:85 , srv1444432982:86 , srv1979391152:86 , srv1980611473:86 , srv267116990:86 , srv19206827:641 }
2023-03-14 23:54:20,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 23:54:20,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1811485332=47, srv1076127282=3, srv1823106232=48, srv1610459948=31, srv1462540522=21, srv272029233=75, srv1951496653=59, srv1670224439=36, srv647332200=93, srv166876104=35, srv1577747257=29, srv945199208=98, srv1916066144=56, srv451842805=85, srv531262236=88, srv1884426529=53, srv1444432982=18, srv479795983=86, srv2092411872=71, srv956556175=99, srv1635616341=33, srv1449743557=20, srv1743253187=41, srv1172935384=7, srv1008083270=0, srv529575854=87, srv1012760030=1, srv1838749328=49, srv1449133167=19, srv326885996=78, srv1297128098=11, srv1317949176=13, srv1843738090=51, srv1563609891=26, srv400543892=82, srv1385642381=14, srv1579804178=30, srv548868610=89, srv1778694112=44, srv568013832=90, srv2090798322=69, srv1101965315=4, srv1416600423=15, srv1480388955=22, srv1561975773=25, srv2000072077=64, srv1114938577=6, srv1677852614=38, srv342849107=79, srv267116990=74, srv1571597471=27, srv1432839964=17, srv1636139619=34, srv214018454=72, srv236153371=73, srv1979391152=62, srv1503303879=23, srv1887453811=54, srv397392210=81, srv1027781054=2, srv684008526=95, srv1247700779=9, srv1762297899=43, srv19206827=57, srv1970739100=60, srv1309239573=12, srv1972106253=61, srv1723836827=40, srv838636639=97, srv2068360164=67, srv2059477250=66, srv2085013793=68, srv369430324=80, srv441785324=83, srv1719822642=39, srv1510606660=24, srv444894315=84, srv1756560127=42, srv681323024=94, srv1575581340=28, srv294997428=76, srv591932855=92, srv1921550511=58, srv314095723=77, srv583465656=91, srv2090891823=70, srv142671848=16, srv812952582=96, srv1625120818=32, srv1792466679=46, srv1787837392=45, srv1980611473=63, srv111474344=5, srv1876082747=52, srv1285448287=10, srv1673251301=37, srv1912262528=55, srv1200312285=8, srv18404025=50, srv2009215686=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 23:54:20,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 23:54:20,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 23:54:20,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 23:54:20,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 23:54:20,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 23:54:20,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 23:54:20,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 23:54:20,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 23:54:20,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 23:54:20,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 23:54:20,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 23:54:20,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 23:54:20,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 23:54:20,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 23:54:20,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 23:54:20,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 23:54:20,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 23:54:20,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 23:54:20,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 23:54:20,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 23:54:20,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 23:54:20,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 23:54:20,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 23:54:20,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 23:54:20,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 23:54:20,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 23:54:20,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 23:54:20,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 23:54:20,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 23:54:20,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 23:54:20,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-14 23:54:20,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-14 23:54:20,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-14 23:54:20,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-14 23:54:20,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-14 23:54:20,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-14 23:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-14 23:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-14 23:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-14 23:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-14 23:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-14 23:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-14 23:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-14 23:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-14 23:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-14 23:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-14 23:54:20,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-14 23:54:20,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-14 23:54:20,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-14 23:54:20,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-14 23:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-14 23:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-14 23:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-14 23:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-14 23:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-14 23:54:20,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-14 23:54:20,542 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.3487020879668901E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024618298195866546); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:54:20,543 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1172935384:83 , srv1970739100:83 , srv2085013793:83 , srv369430324:83 , srv441785324:83 , srv1076127282:84 , srv111474344:84 , srv1200312285:84 , srv1285448287:84 , srv1309239573:84 , srv1317949176:84 , srv1385642381:84 , srv1432839964:84 , srv1449743557:84 , srv1480388955:84 , srv1503303879:84 , srv1563609891:84 , srv1577747257:84 , srv1610459948:84 , srv1625120818:84 , srv1635616341:84 , srv1636139619:84 , srv166876104:84 , srv1670224439:84 , srv1673251301:84 , srv1677852614:84 , srv1719822642:84 , srv1723836827:84 , srv1743253187:84 , srv1778694112:84 , srv1787837392:84 , srv1792466679:84 , srv1811485332:84 , srv1823106232:84 , srv18404025:84 , srv1876082747:84 , srv1884426529:84 , srv1887453811:84 , srv1951496653:84 , srv2090891823:84 , srv2092411872:84 , srv236153371:84 , srv294997428:84 , srv342849107:84 , srv397392210:84 , srv444894315:84 , srv451842805:84 , srv531262236:84 , srv548868610:84 , srv568013832:84 , srv583465656:84 , srv591932855:84 , srv684008526:84 , srv812952582:84 , srv956556175:84 , srv1008083270:85 , srv1012760030:85 , srv1027781054:85 , srv1101965315:85 , srv1114938577:85 , srv1247700779:85 , srv1297128098:85 , srv1416600423:85 , srv142671848:85 , srv1449133167:85 , srv1462540522:85 , srv1510606660:85 , srv1561975773:85 , srv1571597471:85 , srv1575581340:85 , srv1579804178:85 , srv1756560127:85 , srv1762297899:85 , srv1838749328:85 , srv1843738090:85 , srv1912262528:85 , srv1916066144:85 , srv1921550511:85 , srv1972106253:85 , srv2000072077:85 , srv2009215686:85 , srv2059477250:85 , srv2068360164:85 , srv2090798322:85 , srv214018454:85 , srv272029233:85 , srv314095723:85 , srv326885996:85 , srv400543892:85 , srv479795983:85 , srv529575854:85 , srv647332200:85 , srv681323024:85 , srv838636639:85 , srv945199208:85 , srv1444432982:86 , srv1979391152:86 , srv1980611473:86 , srv267116990:86 , srv19206827:641 }
2023-03-14 23:54:20,569 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=270 (was 259) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=3347 (was 3846)
2023-03-14 23:54:20,578 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=270, ProcessCount=169, AvailableMemoryMB=3346
2023-03-14 23:54:20,579 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-14 23:54:20,579 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-14 23:54:20,589 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1174658306:84 , srv1326551529:84 , srv1375421149:84 , srv1394823130:84 , srv1475789251:84 , srv1483333338:84 , srv1512785808:84 , srv1523264863:84 , srv1775436108:84 , srv1846545361:84 , srv1852357282:84 , srv195079129:84 , srv1982134077:84 , srv2032181309:84 , srv2067534746:84 , srv2075577829:84 , srv240123498:84 , srv247049088:84 , srv26568193:84 , srv310677088:84 , srv324380800:84 , srv3680094:84 , srv380299396:84 , srv524577203:84 , srv531535582:84 , srv827290067:84 , srv887742201:84 , srv90653924:84 , srv953315540:84 , srv523061926:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-14 23:54:20,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 23:54:20,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv887742201=27, srv240123498=16, srv1846545361=9, srv2067534746=14, srv90653924=28, srv1394823130=3, srv3680094=21, srv247049088=17, srv2032181309=13, srv380299396=22, srv324380800=20, srv524577203=24, srv827290067=26, srv531535582=25, srv1483333338=5, srv1475789251=4, srv26568193=18, srv523061926=23, srv1375421149=2, srv1174658306=0, srv1775436108=8, srv953315540=29, srv195079129=11, srv1523264863=7, srv2075577829=15, srv1326551529=1, srv1852357282=10, srv310677088=19, srv1982134077=12, srv1512785808=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 23:54:20,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 23:54:20,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 23:54:20,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 23:54:20,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 23:54:20,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 23:54:20,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 23:54:20,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 23:54:20,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 23:54:20,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 23:54:20,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 23:54:20,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 23:54:20,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 23:54:20,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 23:54:20,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 23:54:20,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-14 23:54:20,622 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1371787116006167, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:54:20,622 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912625288905237, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1371787116006167, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-14 23:56:20,608 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6341151 different iterations.  Found a solution that moves 1849 regions; Going from a computed imbalance of 0.9912625288905237 to a new imbalance of 4.3156582404131396E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6848148148148148, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:56:20,615 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1174658306:90 , srv1326551529:90 , srv1375421149:90 , srv1394823130:90 , srv1475789251:90 , srv1483333338:90 , srv1512785808:90 , srv1523264863:90 , srv1775436108:90 , srv1846545361:90 , srv1852357282:90 , srv195079129:90 , srv1982134077:90 , srv2032181309:90 , srv2067534746:90 , srv2075577829:90 , srv240123498:90 , srv247049088:90 , srv26568193:90 , srv310677088:90 , srv324380800:90 , srv3680094:90 , srv380299396:90 , srv523061926:90 , srv524577203:90 , srv531535582:90 , srv827290067:90 , srv887742201:90 , srv90653924:90 , srv953315540:90 }
2023-03-14 23:56:20,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 23:56:20,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv887742201=27, srv240123498=16, srv1846545361=9, srv2067534746=14, srv90653924=28, srv1394823130=3, srv3680094=21, srv247049088=17, srv2032181309=13, srv380299396=22, srv324380800=20, srv524577203=24, srv827290067=26, srv531535582=25, srv1483333338=5, srv1475789251=4, srv26568193=18, srv523061926=23, srv1375421149=2, srv1174658306=0, srv1775436108=8, srv953315540=29, srv195079129=11, srv1523264863=7, srv2075577829=15, srv1326551529=1, srv1852357282=10, srv310677088=19, srv1982134077=12, srv1512785808=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 23:56:20,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 23:56:20,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 23:56:20,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 23:56:20,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 23:56:20,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 23:56:20,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 23:56:20,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 23:56:20,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 23:56:20,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 23:56:20,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-14 23:56:20,634 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 23:56:20,635 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1174658306:90 , srv1326551529:90 , srv1375421149:90 , srv1394823130:90 , srv1475789251:90 , srv1483333338:90 , srv1512785808:90 , srv1523264863:90 , srv1775436108:90 , srv1846545361:90 , srv1852357282:90 , srv195079129:90 , srv1982134077:90 , srv2032181309:90 , srv2067534746:90 , srv2075577829:90 , srv240123498:90 , srv247049088:90 , srv26568193:90 , srv310677088:90 , srv324380800:90 , srv3680094:90 , srv380299396:90 , srv523061926:90 , srv524577203:90 , srv531535582:90 , srv827290067:90 , srv887742201:90 , srv90653924:90 , srv953315540:90 }
2023-03-14 23:56:20,648 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=157 (was 270), ProcessCount=169 (was 169), AvailableMemoryMB=2952 (was 3346)