2023-04-14 00:55:41,287 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-14 00:55:41,614 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-14 00:55:41,733 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-14 00:55:41,791 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-14 00:55:41,798 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-14 00:55:41,811 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=378, ProcessCount=169, AvailableMemoryMB=1370
2023-04-14 00:55:41,813 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-14 00:55:41,813 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-14 00:55:41,929 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1010294747:84 , srv1034089812:84 , srv1041854420:84 , srv1056808713:84 , srv1142110835:84 , srv1150533886:84 , srv1246729139:84 , srv1255762360:84 , srv126734848:84 , srv1320008812:84 , srv1327421309:84 , srv1336856904:84 , srv1356278832:84 , srv1365243552:84 , srv1379297782:84 , srv1399240169:84 , srv1424648552:84 , srv1438731358:84 , srv1461455277:84 , srv1466304337:84 , srv1471352026:84 , srv1492343208:84 , srv1516500093:84 , srv15194390:84 , srv1541386725:84 , srv1555361317:84 , srv1581749633:84 , srv1605674540:84 , srv1622245633:84 , srv1635593766:84 , srv1647286528:84 , srv1682565293:84 , srv1695327643:84 , srv1722059221:84 , srv1745959469:84 , srv1755797489:84 , srv1762722970:84 , srv178706919:84 , srv1843315714:84 , srv1847248387:84 , srv1887331044:84 , srv1908877577:84 , srv1916847729:84 , srv1944215848:84 , srv1950745136:84 , srv197800266:84 , srv2007886997:84 , srv2015709988:84 , srv2023599477:84 , srv2023903853:84 , srv2032000963:84 , srv2043727907:84 , srv2083707947:84 , srv2121586445:84 , srv2125637161:84 , srv213506107:84 , srv216983576:84 , srv229844060:84 , srv236840928:84 , srv240642105:84 , srv2718267:84 , srv285651504:84 , srv31407017:84 , srv320023971:84 , srv355725999:84 , srv374751501:84 , srv407311354:84 , srv425224069:84 , srv443458032:84 , srv459717252:84 , srv463428892:84 , srv465642350:84 , srv479790085:84 , srv497419756:84 , srv559194804:84 , srv607200066:84 , srv646190219:84 , srv647956223:84 , srv659476913:84 , srv665836454:84 , srv69669479:84 , srv718540013:84 , srv731013710:84 , srv73569180:84 , srv754496248:84 , srv768383479:84 , srv770849179:84 , srv782269780:84 , srv807875601:84 , srv808503078:84 , srv820135546:84 , srv820556852:84 , srv862599453:84 , srv865476868:84 , srv885184953:84 , srv887726919:84 , srv916492349:84 , srv935258598:84 , srv98833402:84 , srv875274521:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-14 00:55:41,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 00:55:41,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv820135546=90, srv1424648552=16, srv240642105=59, srv768383479=85, srv1327421309=10, srv2718267=60, srv647956223=77, srv1336856904=11, srv1255762360=7, srv1635593766=29, srv559194804=74, srv216983576=56, srv1379297782=14, srv1365243552=13, srv2032000963=50, srv807875601=88, srv374751501=65, srv2083707947=52, srv2121586445=53, srv875274521=94, srv1466304337=19, srv2023903853=49, srv1916847729=42, srv935258598=98, srv1722059221=33, srv479790085=72, srv718540013=81, srv659476913=78, srv1944215848=43, srv754496248=84, srv820556852=91, srv2125637161=54, srv607200066=75, srv916492349=97, srv285651504=61, srv1950745136=44, srv178706919=37, srv1056808713=3, srv1887331044=40, srv1492343208=21, srv1010294747=0, srv1555361317=25, srv1541386725=24, srv465642350=71, srv407311354=66, srv770849179=86, srv1438731358=17, srv31407017=62, srv443458032=68, srv665836454=79, srv1142110835=4, srv1695327643=32, srv808503078=89, srv497419756=73, srv69669479=80, srv1399240169=15, srv15194390=23, srv1034089812=1, srv236840928=58, srv887726919=96, srv1516500093=22, srv1622245633=28, srv98833402=99, srv1745959469=34, srv197800266=45, srv320023971=63, srv1843315714=38, srv1605674540=27, srv1762722970=36, srv2007886997=46, srv2043727907=51, srv782269780=87, srv126734848=8, srv425224069=67, srv1461455277=18, srv73569180=83, srv1647286528=30, srv459717252=69, srv2023599477=48, srv355725999=64, srv885184953=95, srv1471352026=20, srv229844060=57, srv731013710=82, srv1908877577=41, srv213506107=55, srv1150533886=5, srv865476868=93, srv646190219=76, srv1246729139=6, srv1320008812=9, srv862599453=92, srv1041854420=2, srv463428892=70, srv1356278832=12, srv1581749633=26, srv1682565293=31, srv1755797489=35, srv1847248387=39, srv2015709988=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 00:55:42,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 00:55:42,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 00:55:42,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 00:55:42,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 00:55:42,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 00:55:42,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 00:55:42,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 00:55:42,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 00:55:42,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 00:55:42,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 00:55:42,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 00:55:42,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 00:55:42,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 00:55:42,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 00:55:42,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 00:55:42,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 00:55:42,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 00:55:42,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 00:55:42,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 00:55:42,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 00:55:42,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 00:55:42,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 00:55:42,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 00:55:42,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 00:55:42,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 00:55:42,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 00:55:42,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 00:55:42,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 00:55:42,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 00:55:42,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 00:55:42,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 00:55:42,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 00:55:42,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 00:55:42,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 00:55:42,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 00:55:42,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 00:55:42,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 00:55:42,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 00:55:42,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 00:55:42,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 00:55:42,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 00:55:42,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 00:55:42,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 00:55:42,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 00:55:42,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 00:55:42,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 00:55:42,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 00:55:42,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 00:55:42,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 00:55:42,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 00:55:42,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 00:55:42,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 00:55:42,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 00:55:42,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 00:55:42,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 00:55:42,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 00:55:42,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 00:55:42,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 00:55:42,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 00:55:42,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 00:55:42,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 00:55:42,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 00:55:42,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 00:55:42,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 00:55:42,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 00:55:42,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 00:55:42,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 00:55:42,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 00:55:42,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 00:55:42,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 00:55:42,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 00:55:42,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 00:55:42,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 00:55:42,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 00:55:42,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 00:55:42,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 00:55:42,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 00:55:42,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 00:55:42,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 00:55:42,190 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:42,191 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-14 00:55:42,191 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-14 00:55:42,467 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 403 ms to try 5000 different iterations.  Found a solution that moves 4687 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17925007751364727. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038058637754380634, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05644142201404437, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5207777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038058637754380634, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17433333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:42,481 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv240642105:81 , srv1255762360:82 , srv1320008812:82 , srv1682565293:82 , srv1950745136:82 , srv862599453:82 , srv885184953:82 , srv1010294747:83 , srv1399240169:83 , srv1847248387:83 , srv497419756:83 , srv718540013:83 , srv807875601:83 , srv916492349:83 , srv1246729139:84 , srv1327421309:84 , srv1466304337:84 , srv1516500093:84 , srv15194390:84 , srv2023903853:84 , srv479790085:84 , srv1150533886:85 , srv126734848:85 , srv1336856904:85 , srv1424648552:85 , srv1471352026:85 , srv1492343208:85 , srv1695327643:85 , srv178706919:85 , srv2015709988:85 , srv213506107:85 , srv216983576:85 , srv559194804:85 , srv607200066:85 , srv665836454:85 , srv73569180:85 , srv770849179:85 , srv1461455277:86 , srv1908877577:86 , srv197800266:86 , srv2023599477:86 , srv355725999:86 , srv407311354:86 , srv465642350:86 , srv808503078:86 , srv865476868:86 , srv1056808713:87 , srv1356278832:87 , srv1438731358:87 , srv1581749633:87 , srv1605674540:87 , srv1622245633:87 , srv1722059221:87 , srv2043727907:87 , srv2121586445:87 , srv2125637161:87 , srv285651504:87 , srv443458032:87 , srv731013710:87 , srv754496248:87 , srv768383479:87 , srv887726919:87 , srv935258598:87 , srv1041854420:88 , srv1365243552:88 , srv1379297782:88 , srv1555361317:88 , srv1647286528:88 , srv1755797489:88 , srv1762722970:88 , srv1944215848:88 , srv2032000963:88 , srv229844060:88 , srv374751501:88 , srv459717252:88 , srv647956223:88 , srv69669479:88 , srv782269780:88 , srv1541386725:89 , srv1635593766:89 , srv1745959469:89 , srv1916847729:89 , srv2007886997:89 , srv2083707947:89 , srv320023971:89 , srv659476913:89 , srv820556852:89 , srv98833402:89 , srv1142110835:90 , srv1843315714:90 , srv2718267:90 , srv646190219:90 , srv1887331044:91 , srv31407017:91 , srv425224069:91 , srv820135546:91 , srv236840928:92 , srv1034089812:93 , srv463428892:98 , srv875274521:428 }
2023-04-14 00:55:42,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 00:55:42,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv820135546=90, srv1424648552=16, srv240642105=59, srv768383479=85, srv1327421309=10, srv2718267=60, srv647956223=77, srv1336856904=11, srv1255762360=7, srv1635593766=29, srv559194804=74, srv216983576=56, srv1379297782=14, srv1365243552=13, srv2032000963=50, srv807875601=88, srv374751501=65, srv2083707947=52, srv2121586445=53, srv875274521=94, srv1466304337=19, srv2023903853=49, srv1916847729=42, srv935258598=98, srv1722059221=33, srv479790085=72, srv718540013=81, srv659476913=78, srv1944215848=43, srv754496248=84, srv820556852=91, srv2125637161=54, srv607200066=75, srv916492349=97, srv285651504=61, srv1950745136=44, srv178706919=37, srv1056808713=3, srv1887331044=40, srv1492343208=21, srv1010294747=0, srv1555361317=25, srv1541386725=24, srv465642350=71, srv407311354=66, srv770849179=86, srv1438731358=17, srv31407017=62, srv443458032=68, srv665836454=79, srv1142110835=4, srv1695327643=32, srv808503078=89, srv497419756=73, srv69669479=80, srv1399240169=15, srv15194390=23, srv1034089812=1, srv236840928=58, srv887726919=96, srv1516500093=22, srv1622245633=28, srv98833402=99, srv1745959469=34, srv197800266=45, srv320023971=63, srv1843315714=38, srv1605674540=27, srv1762722970=36, srv2007886997=46, srv2043727907=51, srv782269780=87, srv126734848=8, srv425224069=67, srv1461455277=18, srv73569180=83, srv1647286528=30, srv459717252=69, srv2023599477=48, srv355725999=64, srv885184953=95, srv1471352026=20, srv229844060=57, srv731013710=82, srv1908877577=41, srv213506107=55, srv1150533886=5, srv865476868=93, srv646190219=76, srv1246729139=6, srv1320008812=9, srv862599453=92, srv1041854420=2, srv463428892=70, srv1356278832=12, srv1581749633=26, srv1682565293=31, srv1755797489=35, srv1847248387=39, srv2015709988=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 00:55:42,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 00:55:42,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 00:55:42,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 00:55:42,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 00:55:42,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 00:55:42,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 00:55:42,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 00:55:42,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 00:55:42,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 00:55:42,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 00:55:42,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 00:55:42,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 00:55:42,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 00:55:42,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 00:55:42,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 00:55:42,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 00:55:42,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 00:55:42,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 00:55:42,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 00:55:42,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 00:55:42,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 00:55:42,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 00:55:42,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 00:55:42,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 00:55:42,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 00:55:42,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 00:55:42,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 00:55:42,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 00:55:42,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 00:55:42,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 00:55:42,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 00:55:42,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 00:55:42,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 00:55:42,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 00:55:42,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 00:55:42,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 00:55:42,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 00:55:42,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 00:55:42,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 00:55:42,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 00:55:42,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 00:55:42,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 00:55:42,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 00:55:42,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 00:55:42,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 00:55:42,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 00:55:42,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 00:55:42,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 00:55:42,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 00:55:42,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 00:55:42,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 00:55:42,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 00:55:42,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 00:55:42,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 00:55:42,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 00:55:42,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 00:55:42,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 00:55:42,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 00:55:42,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 00:55:42,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 00:55:42,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 00:55:42,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 00:55:42,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 00:55:42,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 00:55:42,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 00:55:42,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 00:55:42,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 00:55:42,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 00:55:42,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 00:55:42,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 00:55:42,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 00:55:42,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 00:55:42,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 00:55:42,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 00:55:42,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 00:55:42,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 00:55:42,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 00:55:42,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 00:55:42,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 00:55:42,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 00:55:42,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 00:55:42,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 00:55:42,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 00:55:42,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 00:55:42,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 00:55:42,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 00:55:42,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 00:55:42,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 00:55:42,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 00:55:42,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 00:55:42,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 00:55:42,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 00:55:42,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 00:55:42,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 00:55:42,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 00:55:42,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 00:55:42,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 00:55:42,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 00:55:42,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 00:55:42,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 00:55:42,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 00:55:42,636 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038058637754380634, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05644142201404437, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038058637754380634, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17433333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:42,636 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-14 00:55:42,636 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17921725843819108, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038058637754380634, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05644142201404437, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038058637754380634, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17433333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-14 00:55:42,913 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 376 ms to try 5000 different iterations.  Found a solution that moves 2617 regions; Going from a computed imbalance of 0.17921725843819108 to a new imbalance of 0.014950365196704967. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0198063955177214); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0360196978159446, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2907777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0198063955177214); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04633333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:42,925 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv463428892:75 , srv820135546:79 , srv1034089812:80 , srv1887331044:80 , srv236840928:80 , srv31407017:80 , srv1142110835:81 , srv646190219:81 , srv1041854420:82 , srv1635593766:82 , srv1843315714:82 , srv2718267:82 , srv425224069:82 , srv659476913:82 , srv1365243552:83 , srv1379297782:83 , srv1541386725:83 , srv1581749633:83 , srv1647286528:83 , srv820556852:83 , srv1555361317:84 , srv1622245633:84 , srv1745959469:84 , srv1755797489:84 , srv1916847729:84 , srv2007886997:84 , srv2032000963:84 , srv320023971:84 , srv374751501:84 , srv647956223:84 , srv69669479:84 , srv731013710:84 , srv935258598:84 , srv1461455277:85 , srv1492343208:85 , srv1695327643:85 , srv1944215848:85 , srv2023599477:85 , srv2043727907:85 , srv2083707947:85 , srv2121586445:85 , srv285651504:85 , srv355725999:85 , srv443458032:85 , srv459717252:85 , srv768383479:85 , srv1056808713:86 , srv1150533886:86 , srv126734848:86 , srv1438731358:86 , srv1722059221:86 , srv1762722970:86 , srv178706919:86 , srv216983576:86 , srv229844060:86 , srv770849179:86 , srv865476868:86 , srv98833402:86 , srv1246729139:87 , srv1327421309:87 , srv1356278832:87 , srv197800266:87 , srv2015709988:87 , srv2023903853:87 , srv407311354:87 , srv465642350:87 , srv559194804:87 , srv607200066:87 , srv665836454:87 , srv754496248:87 , srv782269780:87 , srv808503078:87 , srv887726919:87 , srv1010294747:88 , srv1336856904:88 , srv1424648552:88 , srv1466304337:88 , srv1471352026:88 , srv1516500093:88 , srv15194390:88 , srv1605674540:88 , srv1950745136:88 , srv2125637161:88 , srv1320008812:89 , srv1399240169:89 , srv1682565293:89 , srv1908877577:89 , srv213506107:89 , srv479790085:89 , srv497419756:89 , srv718540013:89 , srv73569180:89 , srv807875601:89 , srv862599453:89 , srv1255762360:90 , srv1847248387:91 , srv916492349:92 , srv240642105:93 , srv885184953:93 , srv875274521:522 }
2023-04-14 00:55:42,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 00:55:42,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv820135546=90, srv1424648552=16, srv240642105=59, srv768383479=85, srv1327421309=10, srv2718267=60, srv647956223=77, srv1336856904=11, srv1255762360=7, srv1635593766=29, srv559194804=74, srv216983576=56, srv1379297782=14, srv1365243552=13, srv2032000963=50, srv807875601=88, srv374751501=65, srv2083707947=52, srv2121586445=53, srv875274521=94, srv1466304337=19, srv2023903853=49, srv1916847729=42, srv935258598=98, srv1722059221=33, srv479790085=72, srv718540013=81, srv659476913=78, srv1944215848=43, srv754496248=84, srv820556852=91, srv2125637161=54, srv607200066=75, srv916492349=97, srv285651504=61, srv1950745136=44, srv178706919=37, srv1056808713=3, srv1887331044=40, srv1492343208=21, srv1010294747=0, srv1555361317=25, srv1541386725=24, srv465642350=71, srv407311354=66, srv770849179=86, srv1438731358=17, srv31407017=62, srv443458032=68, srv665836454=79, srv1142110835=4, srv1695327643=32, srv808503078=89, srv497419756=73, srv69669479=80, srv1399240169=15, srv15194390=23, srv1034089812=1, srv236840928=58, srv887726919=96, srv1516500093=22, srv1622245633=28, srv98833402=99, srv1745959469=34, srv197800266=45, srv320023971=63, srv1843315714=38, srv1605674540=27, srv1762722970=36, srv2007886997=46, srv2043727907=51, srv782269780=87, srv126734848=8, srv425224069=67, srv1461455277=18, srv73569180=83, srv1647286528=30, srv459717252=69, srv2023599477=48, srv355725999=64, srv885184953=95, srv1471352026=20, srv229844060=57, srv731013710=82, srv1908877577=41, srv213506107=55, srv1150533886=5, srv865476868=93, srv646190219=76, srv1246729139=6, srv1320008812=9, srv862599453=92, srv1041854420=2, srv463428892=70, srv1356278832=12, srv1581749633=26, srv1682565293=31, srv1755797489=35, srv1847248387=39, srv2015709988=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 00:55:42,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 00:55:42,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 00:55:42,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 00:55:42,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 00:55:42,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 00:55:42,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 00:55:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 00:55:42,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 00:55:42,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 00:55:42,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 00:55:42,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 00:55:42,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 00:55:42,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 00:55:42,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 00:55:43,068 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0198063955177214); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0360196978159446, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0198063955177214); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04633333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:43,068 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-14 00:55:43,068 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014932040571044889, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0198063955177214); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0360196978159446, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0198063955177214); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04633333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-14 00:55:43,327 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 356 ms to try 5000 different iterations.  Found a solution that moves 1003 regions; Going from a computed imbalance of 0.014932040571044889 to a new imbalance of 5.899026416426027E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006761392126936675); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012254319653426304); MoveCostFunction : (multiplier=7.0, imbalance=0.11144444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006761392126936675); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:43,332 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2125637161:82 , srv782269780:82 , srv1605674540:83 , srv1847248387:83 , srv1908877577:83 , srv213506107:83 , srv240642105:83 , srv73569180:83 , srv885184953:83 , srv916492349:83 , srv98833402:83 , srv1034089812:84 , srv1056808713:84 , srv1356278832:84 , srv1424648552:84 , srv1438731358:84 , srv1722059221:84 , srv1762722970:84 , srv1916847729:84 , srv1944215848:84 , srv2043727907:84 , srv2083707947:84 , srv229844060:84 , srv407311354:84 , srv425224069:84 , srv463428892:84 , srv465642350:84 , srv479790085:84 , srv754496248:84 , srv808503078:84 , srv887726919:84 , srv1255762360:85 , srv1336856904:85 , srv1399240169:85 , srv1466304337:85 , srv1471352026:85 , srv1516500093:85 , srv15194390:85 , srv1541386725:85 , srv1555361317:85 , srv1745959469:85 , srv1755797489:85 , srv1843315714:85 , srv197800266:85 , srv2007886997:85 , srv2032000963:85 , srv2121586445:85 , srv236840928:85 , srv320023971:85 , srv443458032:85 , srv459717252:85 , srv497419756:85 , srv559194804:85 , srv665836454:85 , srv768383479:85 , srv807875601:85 , srv1010294747:86 , srv1142110835:86 , srv1246729139:86 , srv126734848:86 , srv1320008812:86 , srv1327421309:86 , srv1365243552:86 , srv1379297782:86 , srv1461455277:86 , srv1622245633:86 , srv1635593766:86 , srv1682565293:86 , srv178706919:86 , srv1887331044:86 , srv2015709988:86 , srv2023599477:86 , srv2718267:86 , srv285651504:86 , srv31407017:86 , srv355725999:86 , srv374751501:86 , srv607200066:86 , srv646190219:86 , srv647956223:86 , srv659476913:86 , srv69669479:86 , srv718540013:86 , srv731013710:86 , srv820556852:86 , srv865476868:86 , srv935258598:86 , srv1647286528:87 , srv1695327643:87 , srv1950745136:87 , srv2023903853:87 , srv770849179:87 , srv862599453:87 , srv1041854420:88 , srv1150533886:88 , srv1492343208:88 , srv1581749633:88 , srv216983576:88 , srv820135546:88 , srv875274521:568 }
2023-04-14 00:55:43,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 00:55:43,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv820135546=90, srv1424648552=16, srv240642105=59, srv768383479=85, srv1327421309=10, srv2718267=60, srv647956223=77, srv1336856904=11, srv1255762360=7, srv1635593766=29, srv559194804=74, srv216983576=56, srv1379297782=14, srv1365243552=13, srv2032000963=50, srv807875601=88, srv374751501=65, srv2083707947=52, srv2121586445=53, srv875274521=94, srv1466304337=19, srv2023903853=49, srv1916847729=42, srv935258598=98, srv1722059221=33, srv479790085=72, srv718540013=81, srv659476913=78, srv1944215848=43, srv754496248=84, srv820556852=91, srv2125637161=54, srv607200066=75, srv916492349=97, srv285651504=61, srv1950745136=44, srv178706919=37, srv1056808713=3, srv1887331044=40, srv1492343208=21, srv1010294747=0, srv1555361317=25, srv1541386725=24, srv465642350=71, srv407311354=66, srv770849179=86, srv1438731358=17, srv31407017=62, srv443458032=68, srv665836454=79, srv1142110835=4, srv1695327643=32, srv808503078=89, srv497419756=73, srv69669479=80, srv1399240169=15, srv15194390=23, srv1034089812=1, srv236840928=58, srv887726919=96, srv1516500093=22, srv1622245633=28, srv98833402=99, srv1745959469=34, srv197800266=45, srv320023971=63, srv1843315714=38, srv1605674540=27, srv1762722970=36, srv2007886997=46, srv2043727907=51, srv782269780=87, srv126734848=8, srv425224069=67, srv1461455277=18, srv73569180=83, srv1647286528=30, srv459717252=69, srv2023599477=48, srv355725999=64, srv885184953=95, srv1471352026=20, srv229844060=57, srv731013710=82, srv1908877577=41, srv213506107=55, srv1150533886=5, srv865476868=93, srv646190219=76, srv1246729139=6, srv1320008812=9, srv862599453=92, srv1041854420=2, srv463428892=70, srv1356278832=12, srv1581749633=26, srv1682565293=31, srv1755797489=35, srv1847248387=39, srv2015709988=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 00:55:43,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 00:55:43,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 00:55:43,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 00:55:43,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 00:55:43,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 00:55:43,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 00:55:43,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 00:55:43,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 00:55:43,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 00:55:43,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 00:55:43,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 00:55:43,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 00:55:43,352 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 00:55:43,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 00:55:43,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 00:55:43,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 00:55:43,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 00:55:43,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 00:55:43,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 00:55:43,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 00:55:43,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 00:55:43,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 00:55:43,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 00:55:43,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 00:55:43,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 00:55:43,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 00:55:43,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 00:55:43,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 00:55:43,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 00:55:43,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 00:55:43,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 00:55:43,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 00:55:43,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 00:55:43,354 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 00:55:43,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 00:55:43,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 00:55:43,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 00:55:43,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 00:55:43,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 00:55:43,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 00:55:43,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 00:55:43,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 00:55:43,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 00:55:43,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 00:55:43,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 00:55:43,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 00:55:43,466 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006761392126936675); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012254319653426304); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006761392126936675); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:43,466 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-14 00:55:43,466 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.828794855336773E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006761392126936675); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012254319653426304); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006761392126936675); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-14 00:55:43,839 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 470 ms to try 5000 different iterations.  Found a solution that moves 261 regions; Going from a computed imbalance of 5.828794855336773E-4 to a new imbalance of 3.0129642630159484E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0025070047769678496); MoveCostFunction : (multiplier=7.0, imbalance=0.029, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:43,840 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1150533886:83 , srv1041854420:84 , srv1336856904:84 , srv1466304337:84 , srv1471352026:84 , srv1581749633:84 , srv1605674540:84 , srv1647286528:84 , srv1745959469:84 , srv1762722970:84 , srv1847248387:84 , srv1908877577:84 , srv1916847729:84 , srv2007886997:84 , srv2015709988:84 , srv2023903853:84 , srv2032000963:84 , srv2083707947:84 , srv2125637161:84 , srv216983576:84 , srv229844060:84 , srv2718267:84 , srv285651504:84 , srv320023971:84 , srv374751501:84 , srv459717252:84 , srv463428892:84 , srv607200066:84 , srv647956223:84 , srv659476913:84 , srv69669479:84 , srv718540013:84 , srv754496248:84 , srv770849179:84 , srv820135546:84 , srv820556852:84 , srv862599453:84 , srv865476868:84 , srv885184953:84 , srv887726919:84 , srv916492349:84 , srv98833402:84 , srv1010294747:85 , srv1056808713:85 , srv1142110835:85 , srv1246729139:85 , srv1255762360:85 , srv126734848:85 , srv1320008812:85 , srv1327421309:85 , srv1356278832:85 , srv1365243552:85 , srv1379297782:85 , srv1399240169:85 , srv1424648552:85 , srv1438731358:85 , srv1461455277:85 , srv1492343208:85 , srv1516500093:85 , srv15194390:85 , srv1541386725:85 , srv1555361317:85 , srv1622245633:85 , srv1635593766:85 , srv1682565293:85 , srv1695327643:85 , srv1722059221:85 , srv1755797489:85 , srv178706919:85 , srv1843315714:85 , srv1887331044:85 , srv1944215848:85 , srv197800266:85 , srv2023599477:85 , srv2121586445:85 , srv213506107:85 , srv236840928:85 , srv240642105:85 , srv31407017:85 , srv355725999:85 , srv407311354:85 , srv425224069:85 , srv443458032:85 , srv465642350:85 , srv479790085:85 , srv559194804:85 , srv646190219:85 , srv665836454:85 , srv731013710:85 , srv73569180:85 , srv768383479:85 , srv782269780:85 , srv807875601:85 , srv808503078:85 , srv935258598:85 , srv1034089812:86 , srv1950745136:86 , srv2043727907:86 , srv497419756:86 , srv875274521:624 }
2023-04-14 00:55:43,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 00:55:43,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv820135546=90, srv1424648552=16, srv240642105=59, srv768383479=85, srv1327421309=10, srv2718267=60, srv647956223=77, srv1336856904=11, srv1255762360=7, srv1635593766=29, srv559194804=74, srv216983576=56, srv1379297782=14, srv1365243552=13, srv2032000963=50, srv807875601=88, srv374751501=65, srv2083707947=52, srv2121586445=53, srv875274521=94, srv1466304337=19, srv2023903853=49, srv1916847729=42, srv935258598=98, srv1722059221=33, srv479790085=72, srv718540013=81, srv659476913=78, srv1944215848=43, srv754496248=84, srv820556852=91, srv2125637161=54, srv607200066=75, srv916492349=97, srv285651504=61, srv1950745136=44, srv178706919=37, srv1056808713=3, srv1887331044=40, srv1492343208=21, srv1010294747=0, srv1555361317=25, srv1541386725=24, srv465642350=71, srv407311354=66, srv770849179=86, srv1438731358=17, srv31407017=62, srv443458032=68, srv665836454=79, srv1142110835=4, srv1695327643=32, srv808503078=89, srv497419756=73, srv69669479=80, srv1399240169=15, srv15194390=23, srv1034089812=1, srv236840928=58, srv887726919=96, srv1516500093=22, srv1622245633=28, srv98833402=99, srv1745959469=34, srv197800266=45, srv320023971=63, srv1843315714=38, srv1605674540=27, srv1762722970=36, srv2007886997=46, srv2043727907=51, srv782269780=87, srv126734848=8, srv425224069=67, srv1461455277=18, srv73569180=83, srv1647286528=30, srv459717252=69, srv2023599477=48, srv355725999=64, srv885184953=95, srv1471352026=20, srv229844060=57, srv731013710=82, srv1908877577=41, srv213506107=55, srv1150533886=5, srv865476868=93, srv646190219=76, srv1246729139=6, srv1320008812=9, srv862599453=92, srv1041854420=2, srv463428892=70, srv1356278832=12, srv1581749633=26, srv1682565293=31, srv1755797489=35, srv1847248387=39, srv2015709988=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 00:55:43,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 00:55:43,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 00:55:43,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 00:55:43,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 00:55:43,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 00:55:43,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 00:55:43,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 00:55:43,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 00:55:43,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 00:55:43,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 00:55:43,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 00:55:43,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 00:55:43,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 00:55:43,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 00:55:43,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 00:55:43,970 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0025070047769678496); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:43,970 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-14 00:55:43,971 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.8302081568913676E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0025070047769678496); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-14 00:55:44,190 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 318 ms to try 5000 different iterations.  Found a solution that moves 36 regions; Going from a computed imbalance of 2.8302081568913676E-5 to a new imbalance of 7.941499188108824E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.004); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:44,191 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1034089812:83 , srv1356278832:83 , srv1492343208:83 , srv1950745136:83 , srv197800266:83 , srv216983576:83 , srv497419756:83 , srv1010294747:84 , srv1041854420:84 , srv1056808713:84 , srv1142110835:84 , srv1150533886:84 , srv1246729139:84 , srv1255762360:84 , srv126734848:84 , srv1320008812:84 , srv1327421309:84 , srv1336856904:84 , srv1365243552:84 , srv1379297782:84 , srv1399240169:84 , srv1424648552:84 , srv1438731358:84 , srv1461455277:84 , srv1471352026:84 , srv1516500093:84 , srv15194390:84 , srv1541386725:84 , srv1555361317:84 , srv1581749633:84 , srv1605674540:84 , srv1622245633:84 , srv1635593766:84 , srv1647286528:84 , srv1682565293:84 , srv1695327643:84 , srv1722059221:84 , srv1745959469:84 , srv1755797489:84 , srv1762722970:84 , srv178706919:84 , srv1843315714:84 , srv1887331044:84 , srv1908877577:84 , srv1944215848:84 , srv2007886997:84 , srv2015709988:84 , srv2023599477:84 , srv2023903853:84 , srv2043727907:84 , srv2083707947:84 , srv2121586445:84 , srv213506107:84 , srv229844060:84 , srv236840928:84 , srv240642105:84 , srv2718267:84 , srv285651504:84 , srv31407017:84 , srv320023971:84 , srv355725999:84 , srv374751501:84 , srv407311354:84 , srv425224069:84 , srv443458032:84 , srv459717252:84 , srv465642350:84 , srv479790085:84 , srv559194804:84 , srv607200066:84 , srv646190219:84 , srv647956223:84 , srv665836454:84 , srv69669479:84 , srv718540013:84 , srv731013710:84 , srv73569180:84 , srv754496248:84 , srv768383479:84 , srv770849179:84 , srv782269780:84 , srv807875601:84 , srv808503078:84 , srv820135546:84 , srv820556852:84 , srv862599453:84 , srv865476868:84 , srv885184953:84 , srv887726919:84 , srv916492349:84 , srv935258598:84 , srv98833402:84 , srv1466304337:85 , srv1847248387:85 , srv1916847729:85 , srv2032000963:85 , srv2125637161:85 , srv463428892:85 , srv659476913:85 , srv875274521:684 }
2023-04-14 00:55:44,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 00:55:44,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv820135546=90, srv1424648552=16, srv240642105=59, srv768383479=85, srv1327421309=10, srv2718267=60, srv647956223=77, srv1336856904=11, srv1255762360=7, srv1635593766=29, srv559194804=74, srv216983576=56, srv1379297782=14, srv1365243552=13, srv2032000963=50, srv807875601=88, srv374751501=65, srv2083707947=52, srv2121586445=53, srv875274521=94, srv1466304337=19, srv2023903853=49, srv1916847729=42, srv935258598=98, srv1722059221=33, srv479790085=72, srv718540013=81, srv659476913=78, srv1944215848=43, srv754496248=84, srv820556852=91, srv2125637161=54, srv607200066=75, srv916492349=97, srv285651504=61, srv1950745136=44, srv178706919=37, srv1056808713=3, srv1887331044=40, srv1492343208=21, srv1010294747=0, srv1555361317=25, srv1541386725=24, srv465642350=71, srv407311354=66, srv770849179=86, srv1438731358=17, srv31407017=62, srv443458032=68, srv665836454=79, srv1142110835=4, srv1695327643=32, srv808503078=89, srv497419756=73, srv69669479=80, srv1399240169=15, srv15194390=23, srv1034089812=1, srv236840928=58, srv887726919=96, srv1516500093=22, srv1622245633=28, srv98833402=99, srv1745959469=34, srv197800266=45, srv320023971=63, srv1843315714=38, srv1605674540=27, srv1762722970=36, srv2007886997=46, srv2043727907=51, srv782269780=87, srv126734848=8, srv425224069=67, srv1461455277=18, srv73569180=83, srv1647286528=30, srv459717252=69, srv2023599477=48, srv355725999=64, srv885184953=95, srv1471352026=20, srv229844060=57, srv731013710=82, srv1908877577=41, srv213506107=55, srv1150533886=5, srv865476868=93, srv646190219=76, srv1246729139=6, srv1320008812=9, srv862599453=92, srv1041854420=2, srv463428892=70, srv1356278832=12, srv1581749633=26, srv1682565293=31, srv1755797489=35, srv1847248387=39, srv2015709988=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 00:55:44,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 00:55:44,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 00:55:44,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 00:55:44,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 00:55:44,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 00:55:44,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 00:55:44,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 00:55:44,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 00:55:44,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-14 00:55:44,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-14 00:55:44,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-14 00:55:44,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-14 00:55:44,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-14 00:55:44,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-14 00:55:44,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-14 00:55:44,329 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.689421800350782E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:44,329 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1034089812:83 , srv1356278832:83 , srv1492343208:83 , srv1950745136:83 , srv197800266:83 , srv216983576:83 , srv497419756:83 , srv1010294747:84 , srv1041854420:84 , srv1056808713:84 , srv1142110835:84 , srv1150533886:84 , srv1246729139:84 , srv1255762360:84 , srv126734848:84 , srv1320008812:84 , srv1327421309:84 , srv1336856904:84 , srv1365243552:84 , srv1379297782:84 , srv1399240169:84 , srv1424648552:84 , srv1438731358:84 , srv1461455277:84 , srv1471352026:84 , srv1516500093:84 , srv15194390:84 , srv1541386725:84 , srv1555361317:84 , srv1581749633:84 , srv1605674540:84 , srv1622245633:84 , srv1635593766:84 , srv1647286528:84 , srv1682565293:84 , srv1695327643:84 , srv1722059221:84 , srv1745959469:84 , srv1755797489:84 , srv1762722970:84 , srv178706919:84 , srv1843315714:84 , srv1887331044:84 , srv1908877577:84 , srv1944215848:84 , srv2007886997:84 , srv2015709988:84 , srv2023599477:84 , srv2023903853:84 , srv2043727907:84 , srv2083707947:84 , srv2121586445:84 , srv213506107:84 , srv229844060:84 , srv236840928:84 , srv240642105:84 , srv2718267:84 , srv285651504:84 , srv31407017:84 , srv320023971:84 , srv355725999:84 , srv374751501:84 , srv407311354:84 , srv425224069:84 , srv443458032:84 , srv459717252:84 , srv465642350:84 , srv479790085:84 , srv559194804:84 , srv607200066:84 , srv646190219:84 , srv647956223:84 , srv665836454:84 , srv69669479:84 , srv718540013:84 , srv731013710:84 , srv73569180:84 , srv754496248:84 , srv768383479:84 , srv770849179:84 , srv782269780:84 , srv807875601:84 , srv808503078:84 , srv820135546:84 , srv820556852:84 , srv862599453:84 , srv865476868:84 , srv885184953:84 , srv887726919:84 , srv916492349:84 , srv935258598:84 , srv98833402:84 , srv1466304337:85 , srv1847248387:85 , srv1916847729:85 , srv2032000963:85 , srv2125637161:85 , srv463428892:85 , srv659476913:85 , srv875274521:684 }
2023-04-14 00:55:44,357 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=378 (was 378), ProcessCount=169 (was 169), AvailableMemoryMB=916 (was 1370)
2023-04-14 00:55:44,364 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=378, ProcessCount=169, AvailableMemoryMB=916
2023-04-14 00:55:44,365 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-14 00:55:44,366 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-14 00:55:44,377 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1083212400:84 , srv1119291836:84 , srv1180051358:84 , srv1233704925:84 , srv1427754792:84 , srv1488959125:84 , srv1514250666:84 , srv1528381258:84 , srv1582522126:84 , srv1659722809:84 , srv1829187761:84 , srv1895970984:84 , srv1911182573:84 , srv2066017807:84 , srv2141015947:84 , srv237480061:84 , srv27553363:84 , srv375567620:84 , srv399894305:84 , srv427405529:84 , srv493996544:84 , srv590180096:84 , srv694532561:84 , srv742016956:84 , srv861302455:84 , srv876353407:84 , srv881127369:84 , srv924562919:84 , srv948788596:84 , srv746073288:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-14 00:55:44,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 00:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv746073288=24, srv1895970984=11, srv1233704925=3, srv1514250666=6, srv2066017807=13, srv2141015947=14, srv924562919=28, srv861302455=25, srv1083212400=0, srv399894305=18, srv1829187761=10, srv427405529=19, srv375567620=17, srv27553363=16, srv881127369=27, srv1659722809=9, srv742016956=23, srv1119291836=1, srv1911182573=12, srv1582522126=8, srv237480061=15, srv1528381258=7, srv1488959125=5, srv493996544=20, srv1180051358=2, srv590180096=21, srv876353407=26, srv694532561=22, srv948788596=29, srv1427754792=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 00:55:44,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 00:55:44,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 00:55:44,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 00:55:44,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 00:55:44,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 00:55:44,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 00:55:44,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 00:55:44,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 00:55:44,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 00:55:44,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 00:55:44,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 00:55:44,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 00:55:44,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 00:55:44,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 00:55:44,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 00:55:44,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 00:55:44,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 00:55:44,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-14 00:55:44,420 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0750681349022666, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:55:44,420 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912429580506158, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0750681349022666, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-14 00:57:44,399 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5695200 different iterations.  Found a solution that moves 1854 regions; Going from a computed imbalance of 0.9912429580506158 to a new imbalance of 4.3273284898463826E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6866666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:57:44,405 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1083212400:90 , srv1119291836:90 , srv1180051358:90 , srv1233704925:90 , srv1427754792:90 , srv1488959125:90 , srv1514250666:90 , srv1528381258:90 , srv1582522126:90 , srv1659722809:90 , srv1829187761:90 , srv1895970984:90 , srv1911182573:90 , srv2066017807:90 , srv2141015947:90 , srv237480061:90 , srv27553363:90 , srv375567620:90 , srv399894305:90 , srv427405529:90 , srv493996544:90 , srv590180096:90 , srv694532561:90 , srv742016956:90 , srv746073288:90 , srv861302455:90 , srv876353407:90 , srv881127369:90 , srv924562919:90 , srv948788596:90 }
2023-04-14 00:57:44,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-14 00:57:44,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv746073288=24, srv1895970984=11, srv1233704925=3, srv1514250666=6, srv2066017807=13, srv2141015947=14, srv924562919=28, srv861302455=25, srv1083212400=0, srv399894305=18, srv1829187761=10, srv427405529=19, srv375567620=17, srv27553363=16, srv881127369=27, srv1659722809=9, srv742016956=23, srv1119291836=1, srv1911182573=12, srv1582522126=8, srv237480061=15, srv1528381258=7, srv1488959125=5, srv493996544=20, srv1180051358=2, srv590180096=21, srv876353407=26, srv694532561=22, srv948788596=29, srv1427754792=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-14 00:57:44,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-14 00:57:44,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-14 00:57:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-14 00:57:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-14 00:57:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-14 00:57:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-14 00:57:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-14 00:57:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-14 00:57:44,422 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-14 00:57:44,422 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1083212400:90 , srv1119291836:90 , srv1180051358:90 , srv1233704925:90 , srv1427754792:90 , srv1488959125:90 , srv1514250666:90 , srv1528381258:90 , srv1582522126:90 , srv1659722809:90 , srv1829187761:90 , srv1895970984:90 , srv1911182573:90 , srv2066017807:90 , srv2141015947:90 , srv237480061:90 , srv27553363:90 , srv375567620:90 , srv399894305:90 , srv427405529:90 , srv493996544:90 , srv590180096:90 , srv694532561:90 , srv742016956:90 , srv746073288:90 , srv861302455:90 , srv876353407:90 , srv881127369:90 , srv924562919:90 , srv948788596:90 }
2023-04-14 00:57:44,434 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=275 (was 378), ProcessCount=169 (was 169), AvailableMemoryMB=1098 (was 916) - AvailableMemoryMB LEAK? -