2023-04-26 18:55:39,371 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-26 18:55:39,721 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-26 18:55:39,829 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-26 18:55:39,892 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-26 18:55:39,899 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-26 18:55:39,911 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=371, ProcessCount=170, AvailableMemoryMB=12853
2023-04-26 18:55:39,913 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-26 18:55:39,914 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-26 18:55:40,041 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1030385495:84 , srv1039802292:84 , srv1056503094:84 , srv1067867205:84 , srv1078353058:84 , srv1100559696:84 , srv1145966110:84 , srv1149887750:84 , srv1166377325:84 , srv1172961090:84 , srv1223406117:84 , srv1226813283:84 , srv1231155354:84 , srv1238501315:84 , srv1325315225:84 , srv1397301086:84 , srv1397314015:84 , srv1397450103:84 , srv1408643306:84 , srv1418762967:84 , srv1447170343:84 , srv1456595786:84 , srv1477011972:84 , srv1503490993:84 , srv1546876248:84 , srv1560611326:84 , srv1607224873:84 , srv1633383083:84 , srv1639292324:84 , srv1642795692:84 , srv1651084195:84 , srv1655916760:84 , srv1658260046:84 , srv1669669706:84 , srv1678143793:84 , srv172942111:84 , srv1731828686:84 , srv1738626496:84 , srv1740463400:84 , srv1779708927:84 , srv1822298682:84 , srv1826594028:84 , srv1846754842:84 , srv18559120:84 , srv1913059856:84 , srv194853550:84 , srv1981405790:84 , srv1993093662:84 , srv2002767064:84 , srv2003307878:84 , srv200931086:84 , srv2020302530:84 , srv2087412746:84 , srv2088538026:84 , srv2108097391:84 , srv2119522348:84 , srv2140792548:84 , srv223500726:84 , srv231822662:84 , srv23684879:84 , srv240860115:84 , srv250277426:84 , srv260227080:84 , srv262213896:84 , srv273574648:84 , srv285130657:84 , srv312525775:84 , srv337896360:84 , srv346193346:84 , srv347913483:84 , srv407245082:84 , srv428882303:84 , srv466476725:84 , srv485546204:84 , srv522609180:84 , srv539585686:84 , srv561987930:84 , srv564603330:84 , srv594796101:84 , srv607554128:84 , srv677866460:84 , srv693461940:84 , srv693643298:84 , srv701277095:84 , srv705735420:84 , srv733365191:84 , srv772620223:84 , srv83788289:84 , srv846938826:84 , srv84773724:84 , srv871889575:84 , srv872741518:84 , srv882472135:84 , srv898994818:84 , srv941354194:84 , srv954344942:84 , srv956614310:84 , srv968517034:84 , srv971098542:84 , srv1349784680:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-26 18:55:40,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-26 18:55:40,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2002767064=49, srv250277426=62, srv337896360=68, srv2140792548=57, srv941354194=95, srv733365191=86, srv971098542=99, srv1913059856=45, srv607554128=80, srv240860115=61, srv1226813283=11, srv564603330=78, srv1642795692=30, srv1039802292=1, srv1056503094=2, srv968517034=98, srv1993093662=48, srv1172961090=9, srv1981405790=47, srv701277095=84, srv1822298682=41, srv2119522348=56, srv18559120=44, srv693643298=83, srv954344942=96, srv1738626496=38, srv1231155354=12, srv1418762967=20, srv1030385495=0, srv1223406117=10, srv561987930=77, srv1166377325=8, srv1397301086=16, srv1477011972=23, srv1349784680=15, srv693461940=82, srv346193346=69, srv846938826=89, srv1633383083=28, srv2087412746=53, srv2108097391=55, srv1655916760=32, srv1607224873=27, srv1779708927=40, srv594796101=79, srv1408643306=19, srv485546204=74, srv84773724=90, srv1560611326=26, srv1546876248=25, srv2020302530=52, srv23684879=60, srv1145966110=6, srv705735420=85, srv1397314015=17, srv1100559696=5, srv677866460=81, srv1078353058=4, srv260227080=63, srv428882303=72, srv1669669706=34, srv1503490993=24, srv223500726=58, srv1740463400=39, srv407245082=71, srv1067867205=3, srv1826594028=42, srv262213896=64, srv522609180=75, srv83788289=88, srv1678143793=35, srv273574648=65, srv1658260046=33, srv898994818=94, srv772620223=87, srv1639292324=29, srv231822662=59, srv1447170343=21, srv172942111=36, srv956614310=97, srv2088538026=54, srv1731828686=37, srv1456595786=22, srv1325315225=14, srv1846754842=43, srv1397450103=18, srv871889575=91, srv1149887750=7, srv312525775=67, srv285130657=66, srv1238501315=13, srv539585686=76, srv347913483=70, srv882472135=93, srv466476725=73, srv200931086=51, srv872741518=92, srv1651084195=31, srv2003307878=50, srv194853550=46} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-26 18:55:40,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-26 18:55:40,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-26 18:55:40,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-26 18:55:40,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-26 18:55:40,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-26 18:55:40,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-26 18:55:40,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-26 18:55:40,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-26 18:55:40,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-26 18:55:40,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-26 18:55:40,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-26 18:55:40,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-26 18:55:40,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-26 18:55:40,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-26 18:55:40,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-26 18:55:40,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-26 18:55:40,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-26 18:55:40,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-26 18:55:40,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-26 18:55:40,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-26 18:55:40,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-26 18:55:40,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-26 18:55:40,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-26 18:55:40,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-26 18:55:40,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-26 18:55:40,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-26 18:55:40,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-26 18:55:40,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-26 18:55:40,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-26 18:55:40,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-26 18:55:40,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-26 18:55:40,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-26 18:55:40,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-26 18:55:40,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-26 18:55:40,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-26 18:55:40,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-26 18:55:40,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-26 18:55:40,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-26 18:55:40,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-26 18:55:40,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-26 18:55:40,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-26 18:55:40,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-26 18:55:40,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-26 18:55:40,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-26 18:55:40,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-26 18:55:40,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-26 18:55:40,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-26 18:55:40,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-26 18:55:40,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-26 18:55:40,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-26 18:55:40,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-26 18:55:40,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-26 18:55:40,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-26 18:55:40,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-26 18:55:40,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-26 18:55:40,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-26 18:55:40,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-26 18:55:40,337 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:55:40,337 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-26 18:55:40,337 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-26 18:55:40,679 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 505 ms to try 5000 different iterations.  Found a solution that moves 4685 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17876353542134837. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03767266025037841, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05043692374854567, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5205555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03767266025037841, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.174, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:55:40,700 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1039802292:82 , srv1238501315:83 , srv1397314015:83 , srv872741518:83 , srv968517034:83 , srv1397450103:84 , srv2002767064:84 , srv2087412746:84 , srv2088538026:84 , srv223500726:84 , srv346193346:84 , srv561987930:84 , srv898994818:84 , srv1078353058:85 , srv1166377325:85 , srv1408643306:85 , srv1477011972:85 , srv1669669706:85 , srv1731828686:85 , srv18559120:85 , srv1913059856:85 , srv1993093662:85 , srv200931086:85 , srv2119522348:85 , srv2140792548:85 , srv240860115:85 , srv428882303:85 , srv466476725:85 , srv693643298:85 , srv83788289:85 , srv84773724:85 , srv941354194:85 , srv1172961090:86 , srv1223406117:86 , srv1231155354:86 , srv1418762967:86 , srv1456595786:86 , srv1546876248:86 , srv1560611326:86 , srv1607224873:86 , srv1639292324:86 , srv1642795692:86 , srv172942111:86 , srv1779708927:86 , srv1826594028:86 , srv2003307878:86 , srv2020302530:86 , srv2108097391:86 , srv260227080:86 , srv312525775:86 , srv564603330:86 , srv705735420:86 , srv846938826:86 , srv954344942:86 , srv971098542:86 , srv1447170343:87 , srv1503490993:87 , srv1633383083:87 , srv1738626496:87 , srv194853550:87 , srv250277426:87 , srv262213896:87 , srv337896360:87 , srv347913483:87 , srv594796101:87 , srv733365191:87 , srv956614310:87 , srv1651084195:88 , srv1658260046:88 , srv1678143793:88 , srv23684879:88 , srv407245082:88 , srv485546204:88 , srv539585686:88 , srv693461940:88 , srv701277095:88 , srv772620223:88 , srv871889575:88 , srv882472135:88 , srv1056503094:89 , srv1067867205:89 , srv1100559696:89 , srv1145966110:89 , srv1149887750:89 , srv1226813283:89 , srv1325315225:89 , srv1397301086:89 , srv1655916760:89 , srv1981405790:89 , srv231822662:89 , srv273574648:89 , srv607554128:89 , srv1822298682:90 , srv677866460:90 , srv1030385495:91 , srv1846754842:91 , srv522609180:91 , srv285130657:92 , srv1740463400:94 , srv1349784680:425 }
2023-04-26 18:55:40,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-26 18:55:40,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2002767064=49, srv250277426=62, srv337896360=68, srv2140792548=57, srv941354194=95, srv733365191=86, srv971098542=99, srv1913059856=45, srv607554128=80, srv240860115=61, srv1226813283=11, srv564603330=78, srv1642795692=30, srv1039802292=1, srv1056503094=2, srv968517034=98, srv1993093662=48, srv1172961090=9, srv1981405790=47, srv701277095=84, srv1822298682=41, srv2119522348=56, srv18559120=44, srv693643298=83, srv954344942=96, srv1738626496=38, srv1231155354=12, srv1418762967=20, srv1030385495=0, srv1223406117=10, srv561987930=77, srv1166377325=8, srv1397301086=16, srv1477011972=23, srv1349784680=15, srv693461940=82, srv346193346=69, srv846938826=89, srv1633383083=28, srv2087412746=53, srv2108097391=55, srv1655916760=32, srv1607224873=27, srv1779708927=40, srv594796101=79, srv1408643306=19, srv485546204=74, srv84773724=90, srv1560611326=26, srv1546876248=25, srv2020302530=52, srv23684879=60, srv1145966110=6, srv705735420=85, srv1397314015=17, srv1100559696=5, srv677866460=81, srv1078353058=4, srv260227080=63, srv428882303=72, srv1669669706=34, srv1503490993=24, srv223500726=58, srv1740463400=39, srv407245082=71, srv1067867205=3, srv1826594028=42, srv262213896=64, srv522609180=75, srv83788289=88, srv1678143793=35, srv273574648=65, srv1658260046=33, srv898994818=94, srv772620223=87, srv1639292324=29, srv231822662=59, srv1447170343=21, srv172942111=36, srv956614310=97, srv2088538026=54, srv1731828686=37, srv1456595786=22, srv1325315225=14, srv1846754842=43, srv1397450103=18, srv871889575=91, srv1149887750=7, srv312525775=67, srv285130657=66, srv1238501315=13, srv539585686=76, srv347913483=70, srv882472135=93, srv466476725=73, srv200931086=51, srv872741518=92, srv1651084195=31, srv2003307878=50, srv194853550=46} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-26 18:55:40,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-26 18:55:40,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-26 18:55:40,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-26 18:55:40,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-26 18:55:40,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-26 18:55:40,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-26 18:55:40,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-26 18:55:40,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-26 18:55:40,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-26 18:55:40,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-26 18:55:40,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-26 18:55:40,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-26 18:55:40,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-26 18:55:40,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-26 18:55:40,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-26 18:55:40,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-26 18:55:40,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-26 18:55:40,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-26 18:55:40,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-26 18:55:40,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-26 18:55:40,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-26 18:55:40,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-26 18:55:40,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-26 18:55:40,861 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03767266025037841, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05043692374854567, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03767266025037841, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.174, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:55:40,861 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-26 18:55:40,861 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17873073035019155, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03767266025037841, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05043692374854567, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03767266025037841, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.174, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-26 18:55:41,128 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 388 ms to try 5000 different iterations.  Found a solution that moves 2661 regions; Going from a computed imbalance of 0.17873073035019155 to a new imbalance of 0.015928063135150498. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01958861097704419); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.026717123980786437, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2956666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01958861097704419); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:55:41,138 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1740463400:78 , srv1846754842:80 , srv522609180:81 , srv677866460:81 , srv1056503094:82 , srv1149887750:82 , srv1226813283:82 , srv1325315225:82 , srv1397301086:82 , srv1822298682:82 , srv1981405790:82 , srv407245082:82 , srv1100559696:83 , srv1145966110:83 , srv1658260046:83 , srv273574648:83 , srv882472135:83 , srv1030385495:84 , srv1067867205:84 , srv1655916760:84 , srv1738626496:84 , srv194853550:84 , srv250277426:84 , srv260227080:84 , srv285130657:84 , srv337896360:84 , srv607554128:84 , srv693461940:84 , srv701277095:84 , srv733365191:84 , srv772620223:84 , srv871889575:84 , srv1223406117:85 , srv1231155354:85 , srv1456595786:85 , srv1546876248:85 , srv1560611326:85 , srv1639292324:85 , srv1642795692:85 , srv1651084195:85 , srv172942111:85 , srv1826594028:85 , srv2020302530:85 , srv2108097391:85 , srv23684879:85 , srv347913483:85 , srv594796101:85 , srv971098542:85 , srv1477011972:86 , srv1503490993:86 , srv1607224873:86 , srv1669669706:86 , srv1678143793:86 , srv1779708927:86 , srv200931086:86 , srv2119522348:86 , srv231822662:86 , srv262213896:86 , srv485546204:86 , srv539585686:86 , srv705735420:86 , srv1166377325:87 , srv1172961090:87 , srv1397314015:87 , srv1397450103:87 , srv1408643306:87 , srv1447170343:87 , srv1633383083:87 , srv18559120:87 , srv1993093662:87 , srv240860115:87 , srv466476725:87 , srv564603330:87 , srv83788289:87 , srv84773724:87 , srv898994818:87 , srv956614310:87 , srv1078353058:88 , srv1238501315:88 , srv1418762967:88 , srv1913059856:88 , srv2088538026:88 , srv2140792548:88 , srv223500726:88 , srv561987930:88 , srv846938826:88 , srv954344942:88 , srv1039802292:89 , srv1731828686:89 , srv2003307878:89 , srv2087412746:89 , srv312525775:89 , srv428882303:89 , srv2002767064:90 , srv346193346:90 , srv693643298:90 , srv968517034:90 , srv872741518:91 , srv941354194:91 , srv1349784680:523 }
2023-04-26 18:55:41,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-26 18:55:41,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2002767064=49, srv250277426=62, srv337896360=68, srv2140792548=57, srv941354194=95, srv733365191=86, srv971098542=99, srv1913059856=45, srv607554128=80, srv240860115=61, srv1226813283=11, srv564603330=78, srv1642795692=30, srv1039802292=1, srv1056503094=2, srv968517034=98, srv1993093662=48, srv1172961090=9, srv1981405790=47, srv701277095=84, srv1822298682=41, srv2119522348=56, srv18559120=44, srv693643298=83, srv954344942=96, srv1738626496=38, srv1231155354=12, srv1418762967=20, srv1030385495=0, srv1223406117=10, srv561987930=77, srv1166377325=8, srv1397301086=16, srv1477011972=23, srv1349784680=15, srv693461940=82, srv346193346=69, srv846938826=89, srv1633383083=28, srv2087412746=53, srv2108097391=55, srv1655916760=32, srv1607224873=27, srv1779708927=40, srv594796101=79, srv1408643306=19, srv485546204=74, srv84773724=90, srv1560611326=26, srv1546876248=25, srv2020302530=52, srv23684879=60, srv1145966110=6, srv705735420=85, srv1397314015=17, srv1100559696=5, srv677866460=81, srv1078353058=4, srv260227080=63, srv428882303=72, srv1669669706=34, srv1503490993=24, srv223500726=58, srv1740463400=39, srv407245082=71, srv1067867205=3, srv1826594028=42, srv262213896=64, srv522609180=75, srv83788289=88, srv1678143793=35, srv273574648=65, srv1658260046=33, srv898994818=94, srv772620223=87, srv1639292324=29, srv231822662=59, srv1447170343=21, srv172942111=36, srv956614310=97, srv2088538026=54, srv1731828686=37, srv1456595786=22, srv1325315225=14, srv1846754842=43, srv1397450103=18, srv871889575=91, srv1149887750=7, srv312525775=67, srv285130657=66, srv1238501315=13, srv539585686=76, srv347913483=70, srv882472135=93, srv466476725=73, srv200931086=51, srv872741518=92, srv1651084195=31, srv2003307878=50, srv194853550=46} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-26 18:55:41,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-26 18:55:41,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-26 18:55:41,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-26 18:55:41,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-26 18:55:41,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-26 18:55:41,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-26 18:55:41,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-26 18:55:41,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-26 18:55:41,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-26 18:55:41,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-26 18:55:41,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-26 18:55:41,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-26 18:55:41,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-26 18:55:41,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-26 18:55:41,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-26 18:55:41,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-26 18:55:41,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-26 18:55:41,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-26 18:55:41,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-26 18:55:41,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-26 18:55:41,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-26 18:55:41,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-26 18:55:41,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-26 18:55:41,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-26 18:55:41,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-26 18:55:41,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-26 18:55:41,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-26 18:55:41,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-26 18:55:41,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-26 18:55:41,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-26 18:55:41,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-26 18:55:41,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-26 18:55:41,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-26 18:55:41,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-26 18:55:41,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-26 18:55:41,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-26 18:55:41,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-26 18:55:41,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-26 18:55:41,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-26 18:55:41,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-26 18:55:41,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-26 18:55:41,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-26 18:55:41,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-26 18:55:41,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-26 18:55:41,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-26 18:55:41,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-26 18:55:41,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-26 18:55:41,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-26 18:55:41,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-26 18:55:41,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-26 18:55:41,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-26 18:55:41,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-26 18:55:41,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-26 18:55:41,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-26 18:55:41,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-26 18:55:41,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-26 18:55:41,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-26 18:55:41,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-26 18:55:41,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-26 18:55:41,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-26 18:55:41,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-26 18:55:41,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-26 18:55:41,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-26 18:55:41,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-26 18:55:41,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-26 18:55:41,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-26 18:55:41,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-26 18:55:41,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-26 18:55:41,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-26 18:55:41,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-26 18:55:41,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-26 18:55:41,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-26 18:55:41,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-26 18:55:41,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-26 18:55:41,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-26 18:55:41,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-26 18:55:41,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-26 18:55:41,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-26 18:55:41,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-26 18:55:41,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-26 18:55:41,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-26 18:55:41,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-26 18:55:41,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-26 18:55:41,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-26 18:55:41,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-26 18:55:41,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-26 18:55:41,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-26 18:55:41,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-26 18:55:41,298 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01958861097704419); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.026717123980786437, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01958861097704419); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:55:41,298 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-26 18:55:41,299 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015909430414905383, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01958861097704419); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.026717123980786437, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01958861097704419); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-26 18:55:41,553 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 349 ms to try 5000 different iterations.  Found a solution that moves 994 regions; Going from a computed imbalance of 0.015909430414905383 to a new imbalance of 4.6515038990960455E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005110086268208906); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006321005190772531); MoveCostFunction : (multiplier=7.0, imbalance=0.11044444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005110086268208906); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0028333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:55:41,554 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv285130657:81 , srv312525775:81 , srv1030385495:82 , srv941354194:82 , srv1172961090:83 , srv1633383083:83 , srv1678143793:83 , srv1731828686:83 , srv231822662:83 , srv23684879:83 , srv346193346:83 , srv693643298:83 , srv1408643306:84 , srv1418762967:84 , srv1447170343:84 , srv1651084195:84 , srv1655916760:84 , srv1913059856:84 , srv2002767064:84 , srv2003307878:84 , srv2140792548:84 , srv262213896:84 , srv428882303:84 , srv485546204:84 , srv539585686:84 , srv846938826:84 , srv872741518:84 , srv954344942:84 , srv1067867205:85 , srv1166377325:85 , srv1238501315:85 , srv1397301086:85 , srv1503490993:85 , srv1607224873:85 , srv18559120:85 , srv2088538026:85 , srv223500726:85 , srv240860115:85 , srv522609180:85 , srv607554128:85 , srv693461940:85 , srv705735420:85 , srv83788289:85 , srv956614310:85 , srv968517034:85 , srv1039802292:86 , srv1056503094:86 , srv1078353058:86 , srv1100559696:86 , srv1145966110:86 , srv1223406117:86 , srv1397450103:86 , srv1477011972:86 , srv1546876248:86 , srv1560611326:86 , srv1642795692:86 , srv1669669706:86 , srv172942111:86 , srv1740463400:86 , srv1779708927:86 , srv1981405790:86 , srv1993093662:86 , srv2020302530:86 , srv2087412746:86 , srv2119522348:86 , srv273574648:86 , srv347913483:86 , srv466476725:86 , srv561987930:86 , srv564603330:86 , srv594796101:86 , srv677866460:86 , srv701277095:86 , srv733365191:86 , srv772620223:86 , srv84773724:86 , srv871889575:86 , srv971098542:86 , srv1149887750:87 , srv1226813283:87 , srv1231155354:87 , srv1325315225:87 , srv1397314015:87 , srv1456595786:87 , srv1639292324:87 , srv1658260046:87 , srv1738626496:87 , srv1822298682:87 , srv1826594028:87 , srv1846754842:87 , srv194853550:87 , srv200931086:87 , srv2108097391:87 , srv250277426:87 , srv260227080:87 , srv337896360:87 , srv882472135:87 , srv898994818:87 , srv407245082:88 , srv1349784680:555 }
2023-04-26 18:55:41,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-26 18:55:41,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2002767064=49, srv250277426=62, srv337896360=68, srv2140792548=57, srv941354194=95, srv733365191=86, srv971098542=99, srv1913059856=45, srv607554128=80, srv240860115=61, srv1226813283=11, srv564603330=78, srv1642795692=30, srv1039802292=1, srv1056503094=2, srv968517034=98, srv1993093662=48, srv1172961090=9, srv1981405790=47, srv701277095=84, srv1822298682=41, srv2119522348=56, srv18559120=44, srv693643298=83, srv954344942=96, srv1738626496=38, srv1231155354=12, srv1418762967=20, srv1030385495=0, srv1223406117=10, srv561987930=77, srv1166377325=8, srv1397301086=16, srv1477011972=23, srv1349784680=15, srv693461940=82, srv346193346=69, srv846938826=89, srv1633383083=28, srv2087412746=53, srv2108097391=55, srv1655916760=32, srv1607224873=27, srv1779708927=40, srv594796101=79, srv1408643306=19, srv485546204=74, srv84773724=90, srv1560611326=26, srv1546876248=25, srv2020302530=52, srv23684879=60, srv1145966110=6, srv705735420=85, srv1397314015=17, srv1100559696=5, srv677866460=81, srv1078353058=4, srv260227080=63, srv428882303=72, srv1669669706=34, srv1503490993=24, srv223500726=58, srv1740463400=39, srv407245082=71, srv1067867205=3, srv1826594028=42, srv262213896=64, srv522609180=75, srv83788289=88, srv1678143793=35, srv273574648=65, srv1658260046=33, srv898994818=94, srv772620223=87, srv1639292324=29, srv231822662=59, srv1447170343=21, srv172942111=36, srv956614310=97, srv2088538026=54, srv1731828686=37, srv1456595786=22, srv1325315225=14, srv1846754842=43, srv1397450103=18, srv871889575=91, srv1149887750=7, srv312525775=67, srv285130657=66, srv1238501315=13, srv539585686=76, srv347913483=70, srv882472135=93, srv466476725=73, srv200931086=51, srv872741518=92, srv1651084195=31, srv2003307878=50, srv194853550=46} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-26 18:55:41,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-26 18:55:41,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-26 18:55:41,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-26 18:55:41,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-26 18:55:41,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-26 18:55:41,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-26 18:55:41,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-26 18:55:41,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-26 18:55:41,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-26 18:55:41,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-26 18:55:41,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-26 18:55:41,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-26 18:55:41,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-26 18:55:41,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-26 18:55:41,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-26 18:55:41,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-26 18:55:41,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-26 18:55:41,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-26 18:55:41,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-26 18:55:41,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-26 18:55:41,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-26 18:55:41,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-26 18:55:41,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-26 18:55:41,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-26 18:55:41,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-26 18:55:41,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-26 18:55:41,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-26 18:55:41,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-26 18:55:41,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-26 18:55:41,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-26 18:55:41,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-26 18:55:41,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-26 18:55:41,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-26 18:55:41,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-26 18:55:41,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-26 18:55:41,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-26 18:55:41,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-26 18:55:41,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-26 18:55:41,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-26 18:55:41,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-26 18:55:41,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-26 18:55:41,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-26 18:55:41,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-26 18:55:41,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-26 18:55:41,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-26 18:55:41,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-26 18:55:41,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-26 18:55:41,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-26 18:55:41,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-26 18:55:41,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-26 18:55:41,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-26 18:55:41,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-26 18:55:41,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-26 18:55:41,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-26 18:55:41,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-26 18:55:41,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-26 18:55:41,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-26 18:55:41,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-26 18:55:41,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-26 18:55:41,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-26 18:55:41,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-26 18:55:41,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-26 18:55:41,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-26 18:55:41,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-26 18:55:41,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-26 18:55:41,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-26 18:55:41,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-26 18:55:41,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-26 18:55:41,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-26 18:55:41,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-26 18:55:41,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-26 18:55:41,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-26 18:55:41,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-26 18:55:41,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-26 18:55:41,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-26 18:55:41,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-26 18:55:41,686 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005110086268208906); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006321005190772531); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005110086268208906); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0028333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:55:41,686 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-26 18:55:41,687 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.581902531476186E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005110086268208906); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006321005190772531); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005110086268208906); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0028333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-26 18:55:42,047 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 454 ms to try 5000 different iterations.  Found a solution that moves 197 regions; Going from a computed imbalance of 4.581902531476186E-4 to a new imbalance of 9.973961594745081E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.02188888888888889); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:55:42,048 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1078353058:83 , srv1822298682:83 , srv2003307878:83 , srv2087412746:83 , srv564603330:83 , srv956614310:83 , srv1067867205:84 , srv1100559696:84 , srv1145966110:84 , srv1149887750:84 , srv1226813283:84 , srv1231155354:84 , srv1325315225:84 , srv1418762967:84 , srv1447170343:84 , srv1456595786:84 , srv1503490993:84 , srv1607224873:84 , srv1639292324:84 , srv1658260046:84 , srv1738626496:84 , srv1740463400:84 , srv1826594028:84 , srv1846754842:84 , srv18559120:84 , srv194853550:84 , srv2002767064:84 , srv200931086:84 , srv2108097391:84 , srv231822662:84 , srv250277426:84 , srv262213896:84 , srv273574648:84 , srv337896360:84 , srv347913483:84 , srv407245082:84 , srv428882303:84 , srv466476725:84 , srv485546204:84 , srv539585686:84 , srv561987930:84 , srv594796101:84 , srv607554128:84 , srv693643298:84 , srv701277095:84 , srv772620223:84 , srv846938826:84 , srv84773724:84 , srv871889575:84 , srv872741518:84 , srv898994818:84 , srv954344942:84 , srv968517034:84 , srv1030385495:85 , srv1039802292:85 , srv1056503094:85 , srv1166377325:85 , srv1172961090:85 , srv1223406117:85 , srv1397314015:85 , srv1397450103:85 , srv1477011972:85 , srv1546876248:85 , srv1560611326:85 , srv1633383083:85 , srv1642795692:85 , srv1651084195:85 , srv1655916760:85 , srv1669669706:85 , srv1678143793:85 , srv172942111:85 , srv1731828686:85 , srv1779708927:85 , srv1913059856:85 , srv1981405790:85 , srv1993093662:85 , srv2020302530:85 , srv2088538026:85 , srv2119522348:85 , srv2140792548:85 , srv223500726:85 , srv240860115:85 , srv260227080:85 , srv285130657:85 , srv346193346:85 , srv522609180:85 , srv677866460:85 , srv693461940:85 , srv705735420:85 , srv733365191:85 , srv83788289:85 , srv882472135:85 , srv941354194:85 , srv971098542:85 , srv1238501315:86 , srv1397301086:86 , srv1408643306:86 , srv23684879:86 , srv312525775:86 , srv1349784680:639 }
2023-04-26 18:55:42,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-26 18:55:42,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2002767064=49, srv250277426=62, srv337896360=68, srv2140792548=57, srv941354194=95, srv733365191=86, srv971098542=99, srv1913059856=45, srv607554128=80, srv240860115=61, srv1226813283=11, srv564603330=78, srv1642795692=30, srv1039802292=1, srv1056503094=2, srv968517034=98, srv1993093662=48, srv1172961090=9, srv1981405790=47, srv701277095=84, srv1822298682=41, srv2119522348=56, srv18559120=44, srv693643298=83, srv954344942=96, srv1738626496=38, srv1231155354=12, srv1418762967=20, srv1030385495=0, srv1223406117=10, srv561987930=77, srv1166377325=8, srv1397301086=16, srv1477011972=23, srv1349784680=15, srv693461940=82, srv346193346=69, srv846938826=89, srv1633383083=28, srv2087412746=53, srv2108097391=55, srv1655916760=32, srv1607224873=27, srv1779708927=40, srv594796101=79, srv1408643306=19, srv485546204=74, srv84773724=90, srv1560611326=26, srv1546876248=25, srv2020302530=52, srv23684879=60, srv1145966110=6, srv705735420=85, srv1397314015=17, srv1100559696=5, srv677866460=81, srv1078353058=4, srv260227080=63, srv428882303=72, srv1669669706=34, srv1503490993=24, srv223500726=58, srv1740463400=39, srv407245082=71, srv1067867205=3, srv1826594028=42, srv262213896=64, srv522609180=75, srv83788289=88, srv1678143793=35, srv273574648=65, srv1658260046=33, srv898994818=94, srv772620223=87, srv1639292324=29, srv231822662=59, srv1447170343=21, srv172942111=36, srv956614310=97, srv2088538026=54, srv1731828686=37, srv1456595786=22, srv1325315225=14, srv1846754842=43, srv1397450103=18, srv871889575=91, srv1149887750=7, srv312525775=67, srv285130657=66, srv1238501315=13, srv539585686=76, srv347913483=70, srv882472135=93, srv466476725=73, srv200931086=51, srv872741518=92, srv1651084195=31, srv2003307878=50, srv194853550=46} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-26 18:55:42,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-26 18:55:42,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-26 18:55:42,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-26 18:55:42,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-26 18:55:42,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-26 18:55:42,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-26 18:55:42,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-26 18:55:42,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-26 18:55:42,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-26 18:55:42,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-26 18:55:42,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-26 18:55:42,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-26 18:55:42,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-26 18:55:42,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-26 18:55:42,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-26 18:55:42,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-26 18:55:42,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-26 18:55:42,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-26 18:55:42,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-26 18:55:42,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-26 18:55:42,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-26 18:55:42,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-26 18:55:42,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-26 18:55:42,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-26 18:55:42,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-26 18:55:42,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-26 18:55:42,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-26 18:55:42,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-26 18:55:42,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-26 18:55:42,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-26 18:55:42,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-26 18:55:42,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-26 18:55:42,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-26 18:55:42,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-26 18:55:42,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-26 18:55:42,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-26 18:55:42,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-26 18:55:42,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-26 18:55:42,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-26 18:55:42,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-26 18:55:42,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-26 18:55:42,178 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.594538111735798E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:55:42,179 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1078353058:83 , srv1822298682:83 , srv2003307878:83 , srv2087412746:83 , srv564603330:83 , srv956614310:83 , srv1067867205:84 , srv1100559696:84 , srv1145966110:84 , srv1149887750:84 , srv1226813283:84 , srv1231155354:84 , srv1325315225:84 , srv1418762967:84 , srv1447170343:84 , srv1456595786:84 , srv1503490993:84 , srv1607224873:84 , srv1639292324:84 , srv1658260046:84 , srv1738626496:84 , srv1740463400:84 , srv1826594028:84 , srv1846754842:84 , srv18559120:84 , srv194853550:84 , srv2002767064:84 , srv200931086:84 , srv2108097391:84 , srv231822662:84 , srv250277426:84 , srv262213896:84 , srv273574648:84 , srv337896360:84 , srv347913483:84 , srv407245082:84 , srv428882303:84 , srv466476725:84 , srv485546204:84 , srv539585686:84 , srv561987930:84 , srv594796101:84 , srv607554128:84 , srv693643298:84 , srv701277095:84 , srv772620223:84 , srv846938826:84 , srv84773724:84 , srv871889575:84 , srv872741518:84 , srv898994818:84 , srv954344942:84 , srv968517034:84 , srv1030385495:85 , srv1039802292:85 , srv1056503094:85 , srv1166377325:85 , srv1172961090:85 , srv1223406117:85 , srv1397314015:85 , srv1397450103:85 , srv1477011972:85 , srv1546876248:85 , srv1560611326:85 , srv1633383083:85 , srv1642795692:85 , srv1651084195:85 , srv1655916760:85 , srv1669669706:85 , srv1678143793:85 , srv172942111:85 , srv1731828686:85 , srv1779708927:85 , srv1913059856:85 , srv1981405790:85 , srv1993093662:85 , srv2020302530:85 , srv2088538026:85 , srv2119522348:85 , srv2140792548:85 , srv223500726:85 , srv240860115:85 , srv260227080:85 , srv285130657:85 , srv346193346:85 , srv522609180:85 , srv677866460:85 , srv693461940:85 , srv705735420:85 , srv733365191:85 , srv83788289:85 , srv882472135:85 , srv941354194:85 , srv971098542:85 , srv1238501315:86 , srv1397301086:86 , srv1408643306:86 , srv23684879:86 , srv312525775:86 , srv1349784680:639 }
2023-04-26 18:55:42,207 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=390 (was 371) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=12365 (was 12853)
2023-04-26 18:55:42,217 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=390, ProcessCount=170, AvailableMemoryMB=12365
2023-04-26 18:55:42,218 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-26 18:55:42,218 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-26 18:55:42,227 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1093416018:84 , srv1250356679:84 , srv1298978599:84 , srv1321492101:84 , srv1345807427:84 , srv1412444629:84 , srv1429710542:84 , srv1516532342:84 , srv155691727:84 , srv163760011:84 , srv1805902455:84 , srv1842923019:84 , srv1885228302:84 , srv193418746:84 , srv1974470105:84 , srv269108285:84 , srv311509232:84 , srv312075181:84 , srv381477179:84 , srv477124319:84 , srv533958706:84 , srv566379548:84 , srv668866809:84 , srv832047650:84 , srv886872689:84 , srv944556956:84 , srv967096719:84 , srv970329645:84 , srv999918815:84 , srv1468265106:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-26 18:55:42,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-26 18:55:42,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1842923019=12, srv566379548=22, srv1298978599=2, srv1516532342=8, srv967096719=27, srv1093416018=0, srv381477179=19, srv1429710542=6, srv668866809=23, srv1321492101=3, srv163760011=10, srv999918815=29, srv269108285=16, srv970329645=28, srv193418746=14, srv1250356679=1, srv832047650=24, srv886872689=25, srv944556956=26, srv1468265106=7, srv1805902455=11, srv1885228302=13, srv1412444629=5, srv312075181=18, srv155691727=9, srv1974470105=15, srv533958706=21, srv1345807427=4, srv311509232=17, srv477124319=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-26 18:55:42,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-26 18:55:42,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-26 18:55:42,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-26 18:55:42,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-26 18:55:42,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-26 18:55:42,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-26 18:55:42,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-26 18:55:42,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-26 18:55:42,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-26 18:55:42,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-26 18:55:42,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-26 18:55:42,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-26 18:55:42,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-26 18:55:42,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-26 18:55:42,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-26 18:55:42,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-26 18:55:42,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-26 18:55:42,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-26 18:55:42,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-26 18:55:42,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-26 18:55:42,256 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1146343756173664, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:55:42,257 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912554252438693, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1146343756173664, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-26 18:57:42,243 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5157051 different iterations.  Found a solution that moves 1869 regions; Going from a computed imbalance of 0.9912554252438693 to a new imbalance of 4.362339238146111E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6922222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:57:42,249 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1093416018:90 , srv1250356679:90 , srv1298978599:90 , srv1321492101:90 , srv1345807427:90 , srv1412444629:90 , srv1429710542:90 , srv1468265106:90 , srv1516532342:90 , srv155691727:90 , srv163760011:90 , srv1805902455:90 , srv1842923019:90 , srv1885228302:90 , srv193418746:90 , srv1974470105:90 , srv269108285:90 , srv311509232:90 , srv312075181:90 , srv381477179:90 , srv477124319:90 , srv533958706:90 , srv566379548:90 , srv668866809:90 , srv832047650:90 , srv886872689:90 , srv944556956:90 , srv967096719:90 , srv970329645:90 , srv999918815:90 }
2023-04-26 18:57:42,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-26 18:57:42,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1842923019=12, srv566379548=22, srv1298978599=2, srv1516532342=8, srv967096719=27, srv1093416018=0, srv381477179=19, srv1429710542=6, srv668866809=23, srv1321492101=3, srv163760011=10, srv999918815=29, srv269108285=16, srv970329645=28, srv193418746=14, srv1250356679=1, srv832047650=24, srv886872689=25, srv944556956=26, srv1468265106=7, srv1805902455=11, srv1885228302=13, srv1412444629=5, srv312075181=18, srv155691727=9, srv1974470105=15, srv533958706=21, srv1345807427=4, srv311509232=17, srv477124319=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-26 18:57:42,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-26 18:57:42,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-26 18:57:42,267 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-26 18:57:42,267 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1093416018:90 , srv1250356679:90 , srv1298978599:90 , srv1321492101:90 , srv1345807427:90 , srv1412444629:90 , srv1429710542:90 , srv1468265106:90 , srv1516532342:90 , srv155691727:90 , srv163760011:90 , srv1805902455:90 , srv1842923019:90 , srv1885228302:90 , srv193418746:90 , srv1974470105:90 , srv269108285:90 , srv311509232:90 , srv312075181:90 , srv381477179:90 , srv477124319:90 , srv533958706:90 , srv566379548:90 , srv668866809:90 , srv832047650:90 , srv886872689:90 , srv944556956:90 , srv967096719:90 , srv970329645:90 , srv999918815:90 }
2023-04-26 18:57:42,279 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=336 (was 390), ProcessCount=171 (was 170) - ProcessCount LEAK? -, AvailableMemoryMB=12222 (was 12365)