2023-02-08 00:53:14,282 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-08 00:53:14,595 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-08 00:53:14,710 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-08 00:53:14,773 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-08 00:53:14,780 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-08 00:53:14,793 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=516, ProcessCount=179, AvailableMemoryMB=5027
2023-02-08 00:53:14,795 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-08 00:53:14,796 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-08 00:53:14,933 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv106976649:84 , srv1084224944:84 , srv1087323195:84 , srv1088814750:84 , srv1090715738:84 , srv1104342306:84 , srv1104714198:84 , srv1155207533:84 , srv1161346479:84 , srv1191181028:84 , srv1217011021:84 , srv1226192508:84 , srv126006259:84 , srv1275291713:84 , srv1277740541:84 , srv1290199193:84 , srv1292431879:84 , srv1307290644:84 , srv1331171069:84 , srv1348361407:84 , srv1422047930:84 , srv1425095040:84 , srv1440039159:84 , srv1447989358:84 , srv1492793417:84 , srv1537574967:84 , srv1546949397:84 , srv155502431:84 , srv157608601:84 , srv1578514506:84 , srv1588628441:84 , srv1608919957:84 , srv1623976159:84 , srv1625632760:84 , srv1632645121:84 , srv1647745902:84 , srv1659787524:84 , srv1704855319:84 , srv1711228102:84 , srv1776732665:84 , srv1817960701:84 , srv1820401528:84 , srv1849476663:84 , srv1867374435:84 , srv1873946931:84 , srv1883048740:84 , srv190172578:84 , srv1901730213:84 , srv1940570086:84 , srv1943497175:84 , srv1943913957:84 , srv1950858602:84 , srv1959206479:84 , srv1962905301:84 , srv1987420131:84 , srv2018740817:84 , srv2049268996:84 , srv2062141729:84 , srv2064484225:84 , srv21313506:84 , srv235433332:84 , srv255354403:84 , srv2744806:84 , srv31910389:84 , srv354336029:84 , srv378774124:84 , srv391900435:84 , srv39347425:84 , srv409502117:84 , srv442220717:84 , srv452545740:84 , srv462421332:84 , srv4738107:84 , srv502339512:84 , srv508884377:84 , srv531038558:84 , srv546371625:84 , srv586941737:84 , srv596697725:84 , srv599174638:84 , srv612331127:84 , srv615222493:84 , srv620828916:84 , srv640110996:84 , srv661767987:84 , srv691154935:84 , srv69241668:84 , srv706622585:84 , srv733965921:84 , srv742210033:84 , srv762466259:84 , srv803512512:84 , srv804051615:84 , srv831303612:84 , srv847368914:84 , srv854337905:84 , srv887395387:84 , srv918051525:84 , srv945933166:84 , srv460323732:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-08 00:53:14,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 00:53:14,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1440039159=22, srv1711228102=38, srv1820401528=41, srv599174638=80, srv1217011021=10, srv1776732665=39, srv887395387=97, srv1155207533=7, srv1088814750=3, srv460323732=71, srv1537574967=25, srv706622585=88, srv1425095040=21, srv1275291713=13, srv1867374435=43, srv1084224944=1, srv596697725=79, srv462421332=72, srv409502117=68, srv354336029=64, srv4738107=73, srv1087323195=2, srv2018740817=55, srv661767987=85, srv508884377=75, srv1191181028=9, srv742210033=90, srv1987420131=54, srv733965921=89, srv378774124=65, srv1647745902=35, srv2062141729=57, srv1940570086=48, srv1492793417=24, srv391900435=66, srv69241668=87, srv1348361407=19, srv1943913957=50, srv1292431879=16, srv1422047930=20, srv126006259=12, srv1226192508=11, srv1290199193=15, srv615222493=82, srv1447989358=23, srv39347425=67, srv1623976159=32, srv831303612=94, srv442220717=69, srv854337905=96, srv255354403=61, srv1704855319=37, srv1546949397=26, srv1901730213=47, srv1943497175=49, srv640110996=84, srv1307290644=17, srv1277740541=14, srv1817960701=40, srv2049268996=56, srv612331127=81, srv1588628441=30, srv1883048740=45, srv546371625=77, srv1090715738=4, srv531038558=76, srv1608919957=31, srv1104714198=6, srv1950858602=51, srv235433332=60, srv691154935=86, srv1104342306=5, srv2064484225=58, srv620828916=83, srv1161346479=8, srv1331171069=18, srv106976649=0, srv762466259=91, srv31910389=63, srv1578514506=29, srv1625632760=33, srv1659787524=36, srv2744806=62, srv452545740=70, srv945933166=99, srv1873946931=44, srv190172578=46, srv1632645121=34, srv157608601=28, srv21313506=59, srv1849476663=42, srv918051525=98, srv803512512=92, srv804051615=93, srv847368914=95, srv1962905301=53, srv586941737=78, srv502339512=74, srv155502431=27, srv1959206479=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 00:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 00:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 00:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 00:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 00:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 00:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 00:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 00:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 00:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 00:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 00:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 00:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 00:53:15,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 00:53:15,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 00:53:15,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 00:53:15,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 00:53:15,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 00:53:15,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 00:53:15,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 00:53:15,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 00:53:15,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 00:53:15,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 00:53:15,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 00:53:15,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 00:53:15,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 00:53:15,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 00:53:15,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 00:53:15,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 00:53:15,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 00:53:15,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 00:53:15,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-08 00:53:15,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-08 00:53:15,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-08 00:53:15,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-08 00:53:15,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-08 00:53:15,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-08 00:53:15,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-08 00:53:15,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-08 00:53:15,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-08 00:53:15,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-08 00:53:15,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-08 00:53:15,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-08 00:53:15,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-08 00:53:15,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-08 00:53:15,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-08 00:53:15,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-08 00:53:15,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-08 00:53:15,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-08 00:53:15,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-08 00:53:15,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-08 00:53:15,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-08 00:53:15,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-08 00:53:15,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-08 00:53:15,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-08 00:53:15,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-08 00:53:15,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-08 00:53:15,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-08 00:53:15,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-08 00:53:15,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-08 00:53:15,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-08 00:53:15,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-08 00:53:15,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-08 00:53:15,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-08 00:53:15,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-08 00:53:15,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-08 00:53:15,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-08 00:53:15,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-08 00:53:15,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-08 00:53:15,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-08 00:53:15,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-08 00:53:15,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-08 00:53:15,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-08 00:53:15,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-08 00:53:15,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-08 00:53:15,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-08 00:53:15,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-08 00:53:15,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-08 00:53:15,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-08 00:53:15,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-08 00:53:15,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-08 00:53:15,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-08 00:53:15,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-08 00:53:15,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-08 00:53:15,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-08 00:53:15,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-08 00:53:15,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-08 00:53:15,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-08 00:53:15,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-08 00:53:15,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-08 00:53:15,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-08 00:53:15,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-08 00:53:15,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-08 00:53:15,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-08 00:53:15,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-08 00:53:15,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-08 00:53:15,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-08 00:53:15,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-08 00:53:15,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-08 00:53:15,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-08 00:53:15,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-08 00:53:15,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-08 00:53:15,238 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:53:15,238 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-08 00:53:15,239 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-08 00:53:15,558 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 472 ms to try 5000 different iterations.  Found a solution that moves 4710 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18184598954117664. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04169318273374718, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057542255005440225, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5233333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04169318273374718, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17708333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24358333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:53:15,574 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2064484225:78 , srv409502117:80 , srv1191181028:81 , srv1331171069:81 , srv1588628441:81 , srv1659787524:81 , srv1711228102:81 , srv1962905301:81 , srv190172578:82 , srv502339512:82 , srv586941737:82 , srv640110996:82 , srv1292431879:83 , srv1578514506:83 , srv1623976159:83 , srv1959206479:83 , srv2049268996:83 , srv2744806:83 , srv391900435:83 , srv452545740:83 , srv1307290644:84 , srv2018740817:84 , srv2062141729:84 , srv354336029:84 , srv706622585:84 , srv742210033:84 , srv762466259:84 , srv1090715738:85 , srv1226192508:85 , srv1277740541:85 , srv1704855319:85 , srv1820401528:85 , srv1849476663:85 , srv442220717:85 , srv531038558:85 , srv733965921:85 , srv831303612:85 , srv918051525:85 , srv1104714198:86 , srv1161346479:86 , srv1348361407:86 , srv155502431:86 , srv1608919957:86 , srv1632645121:86 , srv1867374435:86 , srv1873946931:86 , srv1940570086:86 , srv1943913957:86 , srv31910389:86 , srv599174638:86 , srv661767987:86 , srv803512512:86 , srv804051615:86 , srv887395387:86 , srv106976649:87 , srv1084224944:87 , srv1104342306:87 , srv126006259:87 , srv1440039159:87 , srv1447989358:87 , srv1492793417:87 , srv1647745902:87 , srv1901730213:87 , srv255354403:87 , srv378774124:87 , srv546371625:87 , srv615222493:87 , srv847368914:87 , srv1087323195:88 , srv1217011021:88 , srv1422047930:88 , srv1537574967:88 , srv1546949397:88 , srv1883048740:88 , srv1943497175:88 , srv1950858602:88 , srv1987420131:88 , srv235433332:88 , srv612331127:88 , srv691154935:88 , srv69241668:88 , srv1275291713:89 , srv1425095040:89 , srv157608601:89 , srv1625632760:89 , srv1817960701:89 , srv462421332:89 , srv4738107:89 , srv596697725:90 , srv854337905:90 , srv21313506:91 , srv39347425:91 , srv620828916:91 , srv1088814750:92 , srv508884377:92 , srv945933166:94 , srv1776732665:95 , srv1155207533:97 , srv1290199193:97 , srv460323732:460 }
2023-02-08 00:53:15,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 00:53:15,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1440039159=22, srv1711228102=38, srv1820401528=41, srv599174638=80, srv1217011021=10, srv1776732665=39, srv887395387=97, srv1155207533=7, srv1088814750=3, srv460323732=71, srv1537574967=25, srv706622585=88, srv1425095040=21, srv1275291713=13, srv1867374435=43, srv1084224944=1, srv596697725=79, srv462421332=72, srv409502117=68, srv354336029=64, srv4738107=73, srv1087323195=2, srv2018740817=55, srv661767987=85, srv508884377=75, srv1191181028=9, srv742210033=90, srv1987420131=54, srv733965921=89, srv378774124=65, srv1647745902=35, srv2062141729=57, srv1940570086=48, srv1492793417=24, srv391900435=66, srv69241668=87, srv1348361407=19, srv1943913957=50, srv1292431879=16, srv1422047930=20, srv126006259=12, srv1226192508=11, srv1290199193=15, srv615222493=82, srv1447989358=23, srv39347425=67, srv1623976159=32, srv831303612=94, srv442220717=69, srv854337905=96, srv255354403=61, srv1704855319=37, srv1546949397=26, srv1901730213=47, srv1943497175=49, srv640110996=84, srv1307290644=17, srv1277740541=14, srv1817960701=40, srv2049268996=56, srv612331127=81, srv1588628441=30, srv1883048740=45, srv546371625=77, srv1090715738=4, srv531038558=76, srv1608919957=31, srv1104714198=6, srv1950858602=51, srv235433332=60, srv691154935=86, srv1104342306=5, srv2064484225=58, srv620828916=83, srv1161346479=8, srv1331171069=18, srv106976649=0, srv762466259=91, srv31910389=63, srv1578514506=29, srv1625632760=33, srv1659787524=36, srv2744806=62, srv452545740=70, srv945933166=99, srv1873946931=44, srv190172578=46, srv1632645121=34, srv157608601=28, srv21313506=59, srv1849476663=42, srv918051525=98, srv803512512=92, srv804051615=93, srv847368914=95, srv1962905301=53, srv586941737=78, srv502339512=74, srv155502431=27, srv1959206479=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 00:53:15,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 00:53:15,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 00:53:15,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 00:53:15,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 00:53:15,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 00:53:15,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 00:53:15,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 00:53:15,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 00:53:15,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 00:53:15,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 00:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 00:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 00:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 00:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 00:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 00:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 00:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 00:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 00:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 00:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 00:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 00:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 00:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 00:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 00:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 00:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 00:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 00:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 00:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 00:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 00:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-08 00:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-08 00:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-08 00:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-08 00:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-08 00:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-08 00:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-08 00:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-08 00:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-08 00:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-08 00:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-08 00:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-08 00:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-08 00:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-08 00:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-08 00:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-08 00:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-08 00:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-08 00:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-08 00:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-08 00:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-08 00:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-08 00:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-08 00:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-08 00:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-08 00:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-08 00:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-08 00:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-08 00:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-08 00:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-08 00:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-08 00:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-08 00:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-08 00:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-08 00:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-08 00:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-08 00:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-08 00:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-08 00:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-08 00:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-08 00:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-08 00:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-08 00:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-08 00:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-08 00:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-08 00:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-08 00:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-08 00:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-08 00:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-08 00:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-08 00:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-08 00:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-08 00:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-08 00:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-08 00:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-08 00:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-08 00:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-08 00:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-08 00:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-08 00:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-08 00:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-08 00:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-08 00:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-08 00:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-08 00:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-08 00:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-08 00:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-08 00:53:15,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-08 00:53:15,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-08 00:53:15,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-08 00:53:15,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-08 00:53:15,728 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04169318273374718, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057542255005440225, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04169318273374718, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17708333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24358333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:53:15,728 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-08 00:53:15,728 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1818130094162783, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04169318273374718, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057542255005440225, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04169318273374718, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17708333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24358333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-08 00:53:15,924 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 285 ms to try 5000 different iterations.  Found a solution that moves 2620 regions; Going from a computed imbalance of 0.1818130094162783 to a new imbalance of 0.017587642292913794. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02373367213300172); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03769450120210401, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2911111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02373367213300172); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04783333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:53:15,931 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1290199193:74 , srv1155207533:75 , srv1776732665:78 , srv620828916:79 , srv508884377:80 , srv1088814750:81 , srv39347425:81 , srv854337905:81 , srv1625632760:82 , srv1987420131:82 , srv596697725:82 , srv945933166:82 , srv1087323195:83 , srv1447989358:83 , srv1537574967:83 , srv155502431:83 , srv1647745902:83 , srv1817960701:83 , srv1867374435:83 , srv1950858602:83 , srv21313506:83 , srv462421332:83 , srv4738107:83 , srv69241668:83 , srv106976649:84 , srv1217011021:84 , srv1275291713:84 , srv1425095040:84 , srv157608601:84 , srv1608919957:84 , srv1883048740:84 , srv612331127:84 , srv1084224944:85 , srv1104714198:85 , srv1226192508:85 , srv1307290644:85 , srv1422047930:85 , srv1440039159:85 , srv1873946931:85 , srv1901730213:85 , srv1943497175:85 , srv235433332:85 , srv255354403:85 , srv442220717:85 , srv831303612:85 , srv847368914:85 , srv1104342306:86 , srv126006259:86 , srv1546949397:86 , srv1632645121:86 , srv1820401528:86 , srv1849476663:86 , srv1940570086:86 , srv1943913957:86 , srv2744806:86 , srv378774124:86 , srv531038558:86 , srv615222493:86 , srv661767987:86 , srv742210033:86 , srv804051615:86 , srv887395387:86 , srv1161346479:87 , srv1277740541:87 , srv1348361407:87 , srv1492793417:87 , srv1704855319:87 , srv2018740817:87 , srv31910389:87 , srv354336029:87 , srv546371625:87 , srv599174638:87 , srv691154935:87 , srv706622585:87 , srv803512512:87 , srv918051525:87 , srv1623976159:88 , srv2062141729:88 , srv452545740:88 , srv733965921:88 , srv1090715738:89 , srv1191181028:89 , srv1588628441:89 , srv1659787524:89 , srv190172578:89 , srv502339512:89 , srv586941737:89 , srv1292431879:90 , srv1962905301:90 , srv2049268996:90 , srv640110996:90 , srv762466259:90 , srv1331171069:91 , srv1578514506:91 , srv391900435:91 , srv409502117:91 , srv1959206479:92 , srv2064484225:92 , srv1711228102:93 , srv460323732:525 }
2023-02-08 00:53:15,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 00:53:15,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1440039159=22, srv1711228102=38, srv1820401528=41, srv599174638=80, srv1217011021=10, srv1776732665=39, srv887395387=97, srv1155207533=7, srv1088814750=3, srv460323732=71, srv1537574967=25, srv706622585=88, srv1425095040=21, srv1275291713=13, srv1867374435=43, srv1084224944=1, srv596697725=79, srv462421332=72, srv409502117=68, srv354336029=64, srv4738107=73, srv1087323195=2, srv2018740817=55, srv661767987=85, srv508884377=75, srv1191181028=9, srv742210033=90, srv1987420131=54, srv733965921=89, srv378774124=65, srv1647745902=35, srv2062141729=57, srv1940570086=48, srv1492793417=24, srv391900435=66, srv69241668=87, srv1348361407=19, srv1943913957=50, srv1292431879=16, srv1422047930=20, srv126006259=12, srv1226192508=11, srv1290199193=15, srv615222493=82, srv1447989358=23, srv39347425=67, srv1623976159=32, srv831303612=94, srv442220717=69, srv854337905=96, srv255354403=61, srv1704855319=37, srv1546949397=26, srv1901730213=47, srv1943497175=49, srv640110996=84, srv1307290644=17, srv1277740541=14, srv1817960701=40, srv2049268996=56, srv612331127=81, srv1588628441=30, srv1883048740=45, srv546371625=77, srv1090715738=4, srv531038558=76, srv1608919957=31, srv1104714198=6, srv1950858602=51, srv235433332=60, srv691154935=86, srv1104342306=5, srv2064484225=58, srv620828916=83, srv1161346479=8, srv1331171069=18, srv106976649=0, srv762466259=91, srv31910389=63, srv1578514506=29, srv1625632760=33, srv1659787524=36, srv2744806=62, srv452545740=70, srv945933166=99, srv1873946931=44, srv190172578=46, srv1632645121=34, srv157608601=28, srv21313506=59, srv1849476663=42, srv918051525=98, srv803512512=92, srv804051615=93, srv847368914=95, srv1962905301=53, srv586941737=78, srv502339512=74, srv155502431=27, srv1959206479=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 00:53:15,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 00:53:15,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 00:53:15,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 00:53:15,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 00:53:15,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 00:53:15,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-08 00:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-08 00:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-08 00:53:15,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-08 00:53:15,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-08 00:53:15,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-08 00:53:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-08 00:53:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-08 00:53:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-08 00:53:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-08 00:53:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-08 00:53:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-08 00:53:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-08 00:53:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-08 00:53:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-08 00:53:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-08 00:53:16,041 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02373367213300172); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03769450120210401, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02373367213300172); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04783333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:53:16,041 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-08 00:53:16,041 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.017569296660804735, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02373367213300172); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03769450120210401, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02373367213300172); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04783333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-08 00:53:16,206 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 237 ms to try 5000 different iterations.  Found a solution that moves 1057 regions; Going from a computed imbalance of 0.017569296660804735 to a new imbalance of 8.140845227503996E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007796194325812507); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010865982916250669); MoveCostFunction : (multiplier=7.0, imbalance=0.11744444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007796194325812507); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:53:16,208 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1959206479:82 , srv391900435:82 , srv691154935:82 , srv945933166:82 , srv1546949397:83 , srv1711228102:83 , srv21313506:83 , srv378774124:83 , srv546371625:83 , srv762466259:83 , srv1088814750:84 , srv1090715738:84 , srv1161346479:84 , srv126006259:84 , srv1275291713:84 , srv1331171069:84 , srv1348361407:84 , srv1422047930:84 , srv1425095040:84 , srv1440039159:84 , srv157608601:84 , srv1817960701:84 , srv2049268996:84 , srv235433332:84 , srv31910389:84 , srv596697725:84 , srv847368914:84 , srv1104342306:85 , srv1277740541:85 , srv1292431879:85 , srv1492793417:85 , srv1578514506:85 , srv1625632760:85 , srv1704855319:85 , srv1883048740:85 , srv1901730213:85 , srv1940570086:85 , srv1943497175:85 , srv1943913957:85 , srv39347425:85 , srv462421332:85 , srv4738107:85 , srv586941737:85 , srv612331127:85 , srv615222493:85 , srv733965921:85 , srv803512512:85 , srv887395387:85 , srv106976649:86 , srv1084224944:86 , srv1087323195:86 , srv1104714198:86 , srv1155207533:86 , srv1226192508:86 , srv1290199193:86 , srv1447989358:86 , srv1537574967:86 , srv1588628441:86 , srv1608919957:86 , srv1623976159:86 , srv1632645121:86 , srv1776732665:86 , srv1820401528:86 , srv1950858602:86 , srv2062141729:86 , srv2064484225:86 , srv255354403:86 , srv452545740:86 , srv502339512:86 , srv599174638:86 , srv640110996:86 , srv661767987:86 , srv706622585:86 , srv804051615:86 , srv854337905:86 , srv918051525:86 , srv1217011021:87 , srv1647745902:87 , srv1849476663:87 , srv1873946931:87 , srv190172578:87 , srv1962905301:87 , srv2018740817:87 , srv442220717:87 , srv508884377:87 , srv531038558:87 , srv69241668:87 , srv742210033:87 , srv831303612:87 , srv1191181028:88 , srv1307290644:88 , srv155502431:88 , srv1659787524:88 , srv1867374435:88 , srv1987420131:88 , srv2744806:88 , srv354336029:88 , srv409502117:88 , srv620828916:88 , srv460323732:542 }
2023-02-08 00:53:16,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 00:53:16,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1440039159=22, srv1711228102=38, srv1820401528=41, srv599174638=80, srv1217011021=10, srv1776732665=39, srv887395387=97, srv1155207533=7, srv1088814750=3, srv460323732=71, srv1537574967=25, srv706622585=88, srv1425095040=21, srv1275291713=13, srv1867374435=43, srv1084224944=1, srv596697725=79, srv462421332=72, srv409502117=68, srv354336029=64, srv4738107=73, srv1087323195=2, srv2018740817=55, srv661767987=85, srv508884377=75, srv1191181028=9, srv742210033=90, srv1987420131=54, srv733965921=89, srv378774124=65, srv1647745902=35, srv2062141729=57, srv1940570086=48, srv1492793417=24, srv391900435=66, srv69241668=87, srv1348361407=19, srv1943913957=50, srv1292431879=16, srv1422047930=20, srv126006259=12, srv1226192508=11, srv1290199193=15, srv615222493=82, srv1447989358=23, srv39347425=67, srv1623976159=32, srv831303612=94, srv442220717=69, srv854337905=96, srv255354403=61, srv1704855319=37, srv1546949397=26, srv1901730213=47, srv1943497175=49, srv640110996=84, srv1307290644=17, srv1277740541=14, srv1817960701=40, srv2049268996=56, srv612331127=81, srv1588628441=30, srv1883048740=45, srv546371625=77, srv1090715738=4, srv531038558=76, srv1608919957=31, srv1104714198=6, srv1950858602=51, srv235433332=60, srv691154935=86, srv1104342306=5, srv2064484225=58, srv620828916=83, srv1161346479=8, srv1331171069=18, srv106976649=0, srv762466259=91, srv31910389=63, srv1578514506=29, srv1625632760=33, srv1659787524=36, srv2744806=62, srv452545740=70, srv945933166=99, srv1873946931=44, srv190172578=46, srv1632645121=34, srv157608601=28, srv21313506=59, srv1849476663=42, srv918051525=98, srv803512512=92, srv804051615=93, srv847368914=95, srv1962905301=53, srv586941737=78, srv502339512=74, srv155502431=27, srv1959206479=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 00:53:16,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-08 00:53:16,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-08 00:53:16,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-08 00:53:16,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-08 00:53:16,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-08 00:53:16,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-08 00:53:16,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-08 00:53:16,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-08 00:53:16,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-08 00:53:16,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-08 00:53:16,307 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007796194325812507); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010865982916250669); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007796194325812507); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:53:16,307 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-08 00:53:16,307 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.066832505598372E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007796194325812507); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010865982916250669); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007796194325812507); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-08 00:53:16,523 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 291 ms to try 5000 different iterations.  Found a solution that moves 290 regions; Going from a computed imbalance of 8.066832505598372E-4 to a new imbalance of 2.0732486135231134E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.03222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:53:16,524 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1217011021:83 , srv1492793417:83 , srv1578514506:83 , srv1776732665:83 , srv1849476663:83 , srv354336029:83 , srv409502117:83 , srv508884377:83 , srv599174638:83 , srv1084224944:84 , srv1090715738:84 , srv1104342306:84 , srv1155207533:84 , srv1191181028:84 , srv1292431879:84 , srv1632645121:84 , srv1659787524:84 , srv1867374435:84 , srv1873946931:84 , srv190172578:84 , srv1901730213:84 , srv1940570086:84 , srv1962905301:84 , srv1987420131:84 , srv2018740817:84 , srv2062141729:84 , srv255354403:84 , srv531038558:84 , srv546371625:84 , srv615222493:84 , srv620828916:84 , srv640110996:84 , srv661767987:84 , srv69241668:84 , srv733965921:84 , srv803512512:84 , srv918051525:84 , srv945933166:84 , srv106976649:85 , srv1087323195:85 , srv1104714198:85 , srv1161346479:85 , srv1226192508:85 , srv126006259:85 , srv1275291713:85 , srv1290199193:85 , srv1307290644:85 , srv1422047930:85 , srv1425095040:85 , srv1447989358:85 , srv1537574967:85 , srv1546949397:85 , srv155502431:85 , srv157608601:85 , srv1623976159:85 , srv1647745902:85 , srv1711228102:85 , srv1820401528:85 , srv1883048740:85 , srv1943497175:85 , srv1943913957:85 , srv1950858602:85 , srv1959206479:85 , srv2049268996:85 , srv2064484225:85 , srv21313506:85 , srv235433332:85 , srv2744806:85 , srv31910389:85 , srv39347425:85 , srv442220717:85 , srv452545740:85 , srv462421332:85 , srv502339512:85 , srv612331127:85 , srv691154935:85 , srv706622585:85 , srv742210033:85 , srv762466259:85 , srv804051615:85 , srv831303612:85 , srv854337905:85 , srv887395387:85 , srv1088814750:86 , srv1277740541:86 , srv1331171069:86 , srv1348361407:86 , srv1440039159:86 , srv1588628441:86 , srv1608919957:86 , srv1625632760:86 , srv1704855319:86 , srv1817960701:86 , srv378774124:86 , srv391900435:86 , srv4738107:86 , srv586941737:86 , srv596697725:86 , srv847368914:86 , srv460323732:616 }
2023-02-08 00:53:16,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 00:53:16,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1440039159=22, srv1711228102=38, srv1820401528=41, srv599174638=80, srv1217011021=10, srv1776732665=39, srv887395387=97, srv1155207533=7, srv1088814750=3, srv460323732=71, srv1537574967=25, srv706622585=88, srv1425095040=21, srv1275291713=13, srv1867374435=43, srv1084224944=1, srv596697725=79, srv462421332=72, srv409502117=68, srv354336029=64, srv4738107=73, srv1087323195=2, srv2018740817=55, srv661767987=85, srv508884377=75, srv1191181028=9, srv742210033=90, srv1987420131=54, srv733965921=89, srv378774124=65, srv1647745902=35, srv2062141729=57, srv1940570086=48, srv1492793417=24, srv391900435=66, srv69241668=87, srv1348361407=19, srv1943913957=50, srv1292431879=16, srv1422047930=20, srv126006259=12, srv1226192508=11, srv1290199193=15, srv615222493=82, srv1447989358=23, srv39347425=67, srv1623976159=32, srv831303612=94, srv442220717=69, srv854337905=96, srv255354403=61, srv1704855319=37, srv1546949397=26, srv1901730213=47, srv1943497175=49, srv640110996=84, srv1307290644=17, srv1277740541=14, srv1817960701=40, srv2049268996=56, srv612331127=81, srv1588628441=30, srv1883048740=45, srv546371625=77, srv1090715738=4, srv531038558=76, srv1608919957=31, srv1104714198=6, srv1950858602=51, srv235433332=60, srv691154935=86, srv1104342306=5, srv2064484225=58, srv620828916=83, srv1161346479=8, srv1331171069=18, srv106976649=0, srv762466259=91, srv31910389=63, srv1578514506=29, srv1625632760=33, srv1659787524=36, srv2744806=62, srv452545740=70, srv945933166=99, srv1873946931=44, srv190172578=46, srv1632645121=34, srv157608601=28, srv21313506=59, srv1849476663=42, srv918051525=98, srv803512512=92, srv804051615=93, srv847368914=95, srv1962905301=53, srv586941737=78, srv502339512=74, srv155502431=27, srv1959206479=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 00:53:16,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-08 00:53:16,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-08 00:53:16,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-08 00:53:16,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-08 00:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-08 00:53:16,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-08 00:53:16,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-08 00:53:16,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-08 00:53:16,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-08 00:53:16,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-08 00:53:16,619 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.8701862733846913E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:53:16,619 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1217011021:83 , srv1492793417:83 , srv1578514506:83 , srv1776732665:83 , srv1849476663:83 , srv354336029:83 , srv409502117:83 , srv508884377:83 , srv599174638:83 , srv1084224944:84 , srv1090715738:84 , srv1104342306:84 , srv1155207533:84 , srv1191181028:84 , srv1292431879:84 , srv1632645121:84 , srv1659787524:84 , srv1867374435:84 , srv1873946931:84 , srv190172578:84 , srv1901730213:84 , srv1940570086:84 , srv1962905301:84 , srv1987420131:84 , srv2018740817:84 , srv2062141729:84 , srv255354403:84 , srv531038558:84 , srv546371625:84 , srv615222493:84 , srv620828916:84 , srv640110996:84 , srv661767987:84 , srv69241668:84 , srv733965921:84 , srv803512512:84 , srv918051525:84 , srv945933166:84 , srv106976649:85 , srv1087323195:85 , srv1104714198:85 , srv1161346479:85 , srv1226192508:85 , srv126006259:85 , srv1275291713:85 , srv1290199193:85 , srv1307290644:85 , srv1422047930:85 , srv1425095040:85 , srv1447989358:85 , srv1537574967:85 , srv1546949397:85 , srv155502431:85 , srv157608601:85 , srv1623976159:85 , srv1647745902:85 , srv1711228102:85 , srv1820401528:85 , srv1883048740:85 , srv1943497175:85 , srv1943913957:85 , srv1950858602:85 , srv1959206479:85 , srv2049268996:85 , srv2064484225:85 , srv21313506:85 , srv235433332:85 , srv2744806:85 , srv31910389:85 , srv39347425:85 , srv442220717:85 , srv452545740:85 , srv462421332:85 , srv502339512:85 , srv612331127:85 , srv691154935:85 , srv706622585:85 , srv742210033:85 , srv762466259:85 , srv804051615:85 , srv831303612:85 , srv854337905:85 , srv887395387:85 , srv1088814750:86 , srv1277740541:86 , srv1331171069:86 , srv1348361407:86 , srv1440039159:86 , srv1588628441:86 , srv1608919957:86 , srv1625632760:86 , srv1704855319:86 , srv1817960701:86 , srv378774124:86 , srv391900435:86 , srv4738107:86 , srv586941737:86 , srv596697725:86 , srv847368914:86 , srv460323732:616 }
2023-02-08 00:53:16,647 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=514 (was 516), ProcessCount=181 (was 179) - ProcessCount LEAK? -, AvailableMemoryMB=4438 (was 5027)
2023-02-08 00:53:16,656 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=514, ProcessCount=181, AvailableMemoryMB=4436
2023-02-08 00:53:16,657 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-08 00:53:16,657 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-08 00:53:16,668 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1030137808:84 , srv1141445049:84 , srv1205777799:84 , srv1242995664:84 , srv1265700977:84 , srv1331285140:84 , srv1339189031:84 , srv1383500255:84 , srv146287435:84 , srv1478872276:84 , srv152049802:84 , srv1528374218:84 , srv1611384419:84 , srv1625226580:84 , srv167486186:84 , srv1826854990:84 , srv1834616638:84 , srv1867682098:84 , srv194380506:84 , srv1975743117:84 , srv1976377757:84 , srv2054128289:84 , srv326493230:84 , srv368819758:84 , srv429663441:84 , srv512869119:84 , srv815956433:84 , srv819405746:84 , srv887577571:84 , srv1195341188:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-08 00:53:16,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 00:53:16,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv326493230=23, srv368819758=24, srv146287435=9, srv1383500255=8, srv1867682098=18, srv1205777799=3, srv152049802=11, srv1265700977=5, srv1331285140=6, srv1242995664=4, srv194380506=19, srv429663441=25, srv1975743117=20, srv2054128289=22, srv1478872276=10, srv167486186=15, srv1976377757=21, srv512869119=26, srv1339189031=7, srv819405746=28, srv1834616638=17, srv1195341188=2, srv1625226580=14, srv1030137808=0, srv1528374218=12, srv1611384419=13, srv887577571=29, srv815956433=27, srv1141445049=1, srv1826854990=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 00:53:16,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 00:53:16,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 00:53:16,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 00:53:16,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 00:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 00:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 00:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 00:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 00:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 00:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 00:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 00:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 00:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 00:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 00:53:16,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 00:53:16,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 00:53:16,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 00:53:16,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 00:53:16,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 00:53:16,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 00:53:16,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 00:53:16,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 00:53:16,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 00:53:16,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 00:53:16,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 00:53:16,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 00:53:16,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 00:53:16,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 00:53:16,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 00:53:16,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 00:53:16,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-08 00:53:16,711 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.067583108251537, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:53:16,712 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.991240599543159, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.067583108251537, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-08 00:55:16,690 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9260543 different iterations.  Found a solution that moves 1841 regions; Going from a computed imbalance of 0.991240599543159 to a new imbalance of 4.296985841319951E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6818518518518518, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:55:16,696 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1030137808:90 , srv1141445049:90 , srv1195341188:90 , srv1205777799:90 , srv1242995664:90 , srv1265700977:90 , srv1331285140:90 , srv1339189031:90 , srv1383500255:90 , srv146287435:90 , srv1478872276:90 , srv152049802:90 , srv1528374218:90 , srv1611384419:90 , srv1625226580:90 , srv167486186:90 , srv1826854990:90 , srv1834616638:90 , srv1867682098:90 , srv194380506:90 , srv1975743117:90 , srv1976377757:90 , srv2054128289:90 , srv326493230:90 , srv368819758:90 , srv429663441:90 , srv512869119:90 , srv815956433:90 , srv819405746:90 , srv887577571:90 }
2023-02-08 00:55:16,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 00:55:16,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv326493230=23, srv368819758=24, srv146287435=9, srv1383500255=8, srv1867682098=18, srv1205777799=3, srv152049802=11, srv1265700977=5, srv1331285140=6, srv1242995664=4, srv194380506=19, srv429663441=25, srv1975743117=20, srv2054128289=22, srv1478872276=10, srv167486186=15, srv1976377757=21, srv512869119=26, srv1339189031=7, srv819405746=28, srv1834616638=17, srv1195341188=2, srv1625226580=14, srv1030137808=0, srv1528374218=12, srv1611384419=13, srv887577571=29, srv815956433=27, srv1141445049=1, srv1826854990=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 00:55:16,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 00:55:16,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 00:55:16,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 00:55:16,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 00:55:16,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 00:55:16,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 00:55:16,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 00:55:16,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 00:55:16,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 00:55:16,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-08 00:55:16,713 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 00:55:16,713 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1030137808:90 , srv1141445049:90 , srv1195341188:90 , srv1205777799:90 , srv1242995664:90 , srv1265700977:90 , srv1331285140:90 , srv1339189031:90 , srv1383500255:90 , srv146287435:90 , srv1478872276:90 , srv152049802:90 , srv1528374218:90 , srv1611384419:90 , srv1625226580:90 , srv167486186:90 , srv1826854990:90 , srv1834616638:90 , srv1867682098:90 , srv194380506:90 , srv1975743117:90 , srv1976377757:90 , srv2054128289:90 , srv326493230:90 , srv368819758:90 , srv429663441:90 , srv512869119:90 , srv815956433:90 , srv819405746:90 , srv887577571:90 }
2023-02-08 00:55:16,723 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=383 (was 514), ProcessCount=175 (was 181), AvailableMemoryMB=3692 (was 4436)