2024-04-22 20:53:33,388 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-04-22 20:53:33,708 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-04-22 20:53:33,809 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-04-22 20:53:33,867 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-22 20:53:33,875 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-22 20:53:33,889 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=289, MaxFileDescriptor=60000, SystemLoadAverage=307, ProcessCount=170, AvailableMemoryMB=14058
2024-04-22 20:53:33,891 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-22 20:53:33,892 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-22 20:53:34,009 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100661390:84 , srv1025036698:84 , srv1060663830:84 , srv1079829720:84 , srv1088862020:84 , srv1129117392:84 , srv1156854385:84 , srv1180586844:84 , srv1188612978:84 , srv1219084814:84 , srv1228236412:84 , srv1232954078:84 , srv1239308738:84 , srv1245391839:84 , srv1273180577:84 , srv1283402512:84 , srv1300138830:84 , srv1310679201:84 , srv1396672047:84 , srv1470689115:84 , srv1532980815:84 , srv1541622808:84 , srv1542390325:84 , srv1559160519:84 , srv1576255176:84 , srv1592156699:84 , srv160346830:84 , srv1616695046:84 , srv1620559906:84 , srv1659054370:84 , srv1747343783:84 , srv1757379976:84 , srv1794410725:84 , srv1797114143:84 , srv1800728599:84 , srv1803830104:84 , srv1818163130:84 , srv1820981880:84 , srv1830512933:84 , srv1847214032:84 , srv1858042450:84 , srv1879907828:84 , srv1920855422:84 , srv192674803:84 , srv19407699:84 , srv1949472770:84 , srv1968457702:84 , srv1974638136:84 , srv1977884307:84 , srv2006311801:84 , srv2006894373:84 , srv2012054449:84 , srv2037649379:84 , srv2039896996:84 , srv2094228983:84 , srv2143961996:84 , srv237825147:84 , srv250998093:84 , srv302582575:84 , srv312633856:84 , srv322229414:84 , srv337056998:84 , srv347556955:84 , srv380105146:84 , srv389434929:84 , srv416909512:84 , srv429970457:84 , srv435607241:84 , srv478213630:84 , srv521953996:84 , srv563017216:84 , srv600073957:84 , srv606902129:84 , srv609134293:84 , srv609445283:84 , srv613133173:84 , srv618546699:84 , srv621635894:84 , srv638708047:84 , srv646695554:84 , srv659958183:84 , srv66550561:84 , srv676573771:84 , srv686258804:84 , srv698237866:84 , srv730742580:84 , srv732762574:84 , srv734837021:84 , srv787036813:84 , srv79904751:84 , srv815214634:84 , srv8440776:84 , srv887397105:84 , srv920190617:84 , srv966927625:84 , srv977431252:84 , srv983150188:84 , srv996060075:84 , srv996474162:84 , srv179883179:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-04-22 20:53:34,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 20:53:34,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1025036698=1, srv237825147=57, srv996060075=98, srv19407699=45, srv1239308738=12, srv920190617=94, srv732762574=87, srv1800728599=35, srv1232954078=11, srv1300138830=16, srv1794410725=32, srv1879907828=42, srv996474162=99, srv1396672047=18, srv1974638136=48, srv2006894373=51, srv966927625=95, srv380105146=64, srv1156854385=6, srv1245391839=13, srv646695554=80, srv815214634=91, srv1129117392=5, srv1977884307=49, srv416909512=66, srv337056998=62, srv1470689115=19, srv8440776=92, srv977431252=96, srv609134293=74, srv1273180577=14, srv2143961996=56, srv1283402512=15, srv638708047=79, srv322229414=61, srv609445283=75, srv1228236412=10, srv621635894=78, srv734837021=88, srv563017216=71, srv1858042450=41, srv347556955=63, srv1219084814=9, srv600073957=72, srv787036813=89, srv686258804=84, srv1757379976=31, srv1847214032=40, srv2039896996=54, srv698237866=85, srv1616695046=27, srv659958183=81, srv1949472770=46, srv250998093=58, srv1803830104=36, srv160346830=26, srv1532980815=20, srv1620559906=28, srv730742580=86, srv435607241=68, srv389434929=65, srv100661390=0, srv1180586844=7, srv1818163130=37, srv66550561=82, srv1659054370=29, srv1747343783=30, srv606902129=73, srv983150188=97, srv1559160519=23, srv312633856=60, srv521953996=70, srv478213630=69, srv192674803=44, srv179883179=34, srv1541622808=21, srv302582575=59, srv1310679201=17, srv1968457702=47, srv618546699=77, srv1088862020=4, srv1542390325=22, srv1830512933=39, srv887397105=93, srv1576255176=24, srv1079829720=3, srv1920855422=43, srv1188612978=8, srv1820981880=38, srv2006311801=50, srv2012054449=52, srv2037649379=53, srv429970457=67, srv1592156699=25, srv2094228983=55, srv79904751=90, srv1797114143=33, srv676573771=83, srv1060663830=2, srv613133173=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 20:53:34,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 20:53:34,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 20:53:34,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 20:53:34,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 20:53:34,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 20:53:34,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 20:53:34,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 20:53:34,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 20:53:34,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 20:53:34,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 20:53:34,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 20:53:34,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 20:53:34,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 20:53:34,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 20:53:34,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 20:53:34,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 20:53:34,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 20:53:34,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 20:53:34,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 20:53:34,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-22 20:53:34,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-22 20:53:34,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-22 20:53:34,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-22 20:53:34,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-22 20:53:34,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-22 20:53:34,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-22 20:53:34,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-22 20:53:34,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-22 20:53:34,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-22 20:53:34,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-22 20:53:34,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-22 20:53:34,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-22 20:53:34,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-22 20:53:34,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-22 20:53:34,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-22 20:53:34,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-22 20:53:34,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-22 20:53:34,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-22 20:53:34,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-22 20:53:34,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-22 20:53:34,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-22 20:53:34,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-22 20:53:34,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-22 20:53:34,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-22 20:53:34,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-22 20:53:34,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-22 20:53:34,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-22 20:53:34,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-22 20:53:34,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-22 20:53:34,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-22 20:53:34,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-22 20:53:34,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-22 20:53:34,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-22 20:53:34,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-22 20:53:34,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-22 20:53:34,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-22 20:53:34,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-22 20:53:34,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-22 20:53:34,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-22 20:53:34,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-22 20:53:34,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-22 20:53:34,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-22 20:53:34,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-22 20:53:34,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-22 20:53:34,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-22 20:53:34,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-22 20:53:34,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-22 20:53:34,371 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:53:34,371 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-22 20:53:34,372 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-22 20:53:34,725 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 600 ms to try 5000 different iterations.  Found a solution that moves 4659 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18599556900305725. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039413963658052824, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05357269519105483, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5176666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039413963658052824, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18133333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:53:34,737 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1079829720:81 , srv732762574:81 , srv100661390:82 , srv1820981880:82 , srv1920855422:82 , srv2006894373:82 , srv618546699:82 , srv815214634:82 , srv1541622808:83 , srv1794410725:83 , srv1974638136:83 , srv429970457:83 , srv435607241:83 , srv646695554:83 , srv1797114143:84 , srv1800728599:84 , srv2094228983:84 , srv734837021:84 , srv977431252:84 , srv1129117392:85 , srv1180586844:85 , srv1228236412:85 , srv1245391839:85 , srv1616695046:85 , srv1830512933:85 , srv1858042450:85 , srv2037649379:85 , srv2039896996:85 , srv302582575:85 , srv312633856:85 , srv337056998:85 , srv600073957:85 , srv609445283:85 , srv621635894:85 , srv676573771:85 , srv698237866:85 , srv730742580:85 , srv8440776:85 , srv966927625:85 , srv996060075:85 , srv1273180577:86 , srv160346830:86 , srv1757379976:86 , srv1803830104:86 , srv1818163130:86 , srv1968457702:86 , srv347556955:86 , srv389434929:86 , srv521953996:86 , srv563017216:86 , srv606902129:86 , srv659958183:86 , srv686258804:86 , srv983150188:86 , srv996474162:86 , srv1156854385:87 , srv1188612978:87 , srv1232954078:87 , srv1283402512:87 , srv1310679201:87 , srv1396672047:87 , srv1592156699:87 , srv1879907828:87 , srv192674803:87 , srv19407699:87 , srv1949472770:87 , srv322229414:87 , srv787036813:87 , srv1025036698:88 , srv1060663830:88 , srv1559160519:88 , srv1576255176:88 , srv1977884307:88 , srv2012054449:88 , srv250998093:88 , srv416909512:88 , srv478213630:88 , srv609134293:88 , srv66550561:88 , srv79904751:88 , srv1219084814:89 , srv1239308738:89 , srv1532980815:89 , srv1847214032:89 , srv638708047:89 , srv887397105:89 , srv1088862020:90 , srv1542390325:90 , srv1620559906:90 , srv2143961996:90 , srv237825147:90 , srv380105146:90 , srv1300138830:91 , srv1659054370:91 , srv613133173:91 , srv1470689115:92 , srv920190617:93 , srv2006311801:96 , srv1747343783:98 , srv179883179:440 }
2024-04-22 20:53:34,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 20:53:34,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1025036698=1, srv237825147=57, srv996060075=98, srv19407699=45, srv1239308738=12, srv920190617=94, srv732762574=87, srv1800728599=35, srv1232954078=11, srv1300138830=16, srv1794410725=32, srv1879907828=42, srv996474162=99, srv1396672047=18, srv1974638136=48, srv2006894373=51, srv966927625=95, srv380105146=64, srv1156854385=6, srv1245391839=13, srv646695554=80, srv815214634=91, srv1129117392=5, srv1977884307=49, srv416909512=66, srv337056998=62, srv1470689115=19, srv8440776=92, srv977431252=96, srv609134293=74, srv1273180577=14, srv2143961996=56, srv1283402512=15, srv638708047=79, srv322229414=61, srv609445283=75, srv1228236412=10, srv621635894=78, srv734837021=88, srv563017216=71, srv1858042450=41, srv347556955=63, srv1219084814=9, srv600073957=72, srv787036813=89, srv686258804=84, srv1757379976=31, srv1847214032=40, srv2039896996=54, srv698237866=85, srv1616695046=27, srv659958183=81, srv1949472770=46, srv250998093=58, srv1803830104=36, srv160346830=26, srv1532980815=20, srv1620559906=28, srv730742580=86, srv435607241=68, srv389434929=65, srv100661390=0, srv1180586844=7, srv1818163130=37, srv66550561=82, srv1659054370=29, srv1747343783=30, srv606902129=73, srv983150188=97, srv1559160519=23, srv312633856=60, srv521953996=70, srv478213630=69, srv192674803=44, srv179883179=34, srv1541622808=21, srv302582575=59, srv1310679201=17, srv1968457702=47, srv618546699=77, srv1088862020=4, srv1542390325=22, srv1830512933=39, srv887397105=93, srv1576255176=24, srv1079829720=3, srv1920855422=43, srv1188612978=8, srv1820981880=38, srv2006311801=50, srv2012054449=52, srv2037649379=53, srv429970457=67, srv1592156699=25, srv2094228983=55, srv79904751=90, srv1797114143=33, srv676573771=83, srv1060663830=2, srv613133173=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 20:53:34,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 20:53:34,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 20:53:34,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 20:53:34,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 20:53:34,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 20:53:34,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 20:53:34,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 20:53:34,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 20:53:34,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 20:53:34,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 20:53:34,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 20:53:34,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 20:53:34,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 20:53:34,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 20:53:34,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 20:53:34,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 20:53:34,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 20:53:34,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 20:53:34,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 20:53:34,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 20:53:34,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 20:53:34,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 20:53:34,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 20:53:34,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 20:53:34,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 20:53:34,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 20:53:34,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 20:53:34,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 20:53:34,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 20:53:34,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 20:53:34,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-22 20:53:34,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-22 20:53:34,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-22 20:53:34,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-22 20:53:34,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-22 20:53:34,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-22 20:53:34,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-22 20:53:34,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-22 20:53:34,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-22 20:53:34,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-22 20:53:34,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-22 20:53:34,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-22 20:53:34,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-22 20:53:34,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-22 20:53:34,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-22 20:53:34,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-22 20:53:34,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-22 20:53:34,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-22 20:53:34,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-22 20:53:34,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-22 20:53:34,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-22 20:53:34,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-22 20:53:34,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-22 20:53:34,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-22 20:53:34,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-22 20:53:34,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-22 20:53:34,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-22 20:53:34,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-22 20:53:34,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-22 20:53:34,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-22 20:53:34,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-22 20:53:34,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-22 20:53:34,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-22 20:53:34,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-22 20:53:34,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-22 20:53:34,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-22 20:53:34,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-22 20:53:34,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-22 20:53:34,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-22 20:53:34,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-22 20:53:34,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-22 20:53:34,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-22 20:53:34,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-22 20:53:34,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-22 20:53:34,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-22 20:53:34,942 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039413963658052824, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05357269519105483, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039413963658052824, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18133333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:53:34,943 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-22 20:53:34,943 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18596294598779153, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039413963658052824, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05357269519105483, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039413963658052824, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18133333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-22 20:53:35,186 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 382 ms to try 5000 different iterations.  Found a solution that moves 2662 regions; Going from a computed imbalance of 0.18596294598779153 to a new imbalance of 0.013867874433301303. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021052254182399782); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03247030932489435, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29577777777777775, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021052254182399782); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:53:35,198 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1747343783:74 , srv2006311801:75 , srv920190617:78 , srv237825147:79 , srv380105146:80 , srv613133173:80 , srv1300138830:81 , srv1542390325:81 , srv1060663830:82 , srv1239308738:82 , srv1470689115:82 , srv1532980815:82 , srv1659054370:82 , srv250998093:82 , srv66550561:82 , srv887397105:82 , srv1088862020:83 , srv1620559906:83 , srv1847214032:83 , srv1879907828:83 , srv1949472770:83 , srv2143961996:83 , srv416909512:83 , srv478213630:83 , srv609134293:83 , srv1025036698:84 , srv1156854385:84 , srv1188612978:84 , srv1283402512:84 , srv1818163130:84 , srv1968457702:84 , srv1977884307:84 , srv638708047:84 , srv79904751:84 , srv1219084814:85 , srv1310679201:85 , srv1559160519:85 , srv1576255176:85 , srv1803830104:85 , srv192674803:85 , srv2012054449:85 , srv606902129:85 , srv609445283:85 , srv983150188:85 , srv996474162:85 , srv1228236412:86 , srv1232954078:86 , srv1245391839:86 , srv1273180577:86 , srv1396672047:86 , srv1592156699:86 , srv160346830:86 , srv19407699:86 , srv2039896996:86 , srv322229414:86 , srv347556955:86 , srv563017216:86 , srv659958183:86 , srv730742580:86 , srv787036813:86 , srv996060075:86 , srv1541622808:87 , srv1757379976:87 , srv1858042450:87 , srv2037649379:87 , srv2094228983:87 , srv8440776:87 , srv1180586844:88 , srv1616695046:88 , srv1830512933:88 , srv2006894373:88 , srv312633856:88 , srv389434929:88 , srv521953996:88 , srv600073957:88 , srv621635894:88 , srv646695554:88 , srv734837021:88 , srv815214634:88 , srv100661390:89 , srv1129117392:89 , srv1794410725:89 , srv1800728599:89 , srv337056998:89 , srv435607241:89 , srv676573771:89 , srv686258804:89 , srv1797114143:90 , srv1920855422:90 , srv429970457:90 , srv618546699:90 , srv966927625:90 , srv302582575:91 , srv732762574:91 , srv977431252:91 , srv1079829720:92 , srv1820981880:92 , srv1974638136:92 , srv698237866:92 , srv179883179:521 }
2024-04-22 20:53:35,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 20:53:35,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1025036698=1, srv237825147=57, srv996060075=98, srv19407699=45, srv1239308738=12, srv920190617=94, srv732762574=87, srv1800728599=35, srv1232954078=11, srv1300138830=16, srv1794410725=32, srv1879907828=42, srv996474162=99, srv1396672047=18, srv1974638136=48, srv2006894373=51, srv966927625=95, srv380105146=64, srv1156854385=6, srv1245391839=13, srv646695554=80, srv815214634=91, srv1129117392=5, srv1977884307=49, srv416909512=66, srv337056998=62, srv1470689115=19, srv8440776=92, srv977431252=96, srv609134293=74, srv1273180577=14, srv2143961996=56, srv1283402512=15, srv638708047=79, srv322229414=61, srv609445283=75, srv1228236412=10, srv621635894=78, srv734837021=88, srv563017216=71, srv1858042450=41, srv347556955=63, srv1219084814=9, srv600073957=72, srv787036813=89, srv686258804=84, srv1757379976=31, srv1847214032=40, srv2039896996=54, srv698237866=85, srv1616695046=27, srv659958183=81, srv1949472770=46, srv250998093=58, srv1803830104=36, srv160346830=26, srv1532980815=20, srv1620559906=28, srv730742580=86, srv435607241=68, srv389434929=65, srv100661390=0, srv1180586844=7, srv1818163130=37, srv66550561=82, srv1659054370=29, srv1747343783=30, srv606902129=73, srv983150188=97, srv1559160519=23, srv312633856=60, srv521953996=70, srv478213630=69, srv192674803=44, srv179883179=34, srv1541622808=21, srv302582575=59, srv1310679201=17, srv1968457702=47, srv618546699=77, srv1088862020=4, srv1542390325=22, srv1830512933=39, srv887397105=93, srv1576255176=24, srv1079829720=3, srv1920855422=43, srv1188612978=8, srv1820981880=38, srv2006311801=50, srv2012054449=52, srv2037649379=53, srv429970457=67, srv1592156699=25, srv2094228983=55, srv79904751=90, srv1797114143=33, srv676573771=83, srv1060663830=2, srv613133173=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 20:53:35,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 20:53:35,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 20:53:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 20:53:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 20:53:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 20:53:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 20:53:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 20:53:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 20:53:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 20:53:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 20:53:35,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 20:53:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 20:53:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 20:53:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 20:53:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 20:53:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 20:53:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 20:53:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 20:53:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 20:53:35,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 20:53:35,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 20:53:35,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 20:53:35,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 20:53:35,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 20:53:35,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 20:53:35,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 20:53:35,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 20:53:35,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 20:53:35,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 20:53:35,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 20:53:35,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-22 20:53:35,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-22 20:53:35,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-22 20:53:35,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-22 20:53:35,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-22 20:53:35,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-22 20:53:35,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-22 20:53:35,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-22 20:53:35,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-22 20:53:35,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-22 20:53:35,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-22 20:53:35,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-22 20:53:35,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-22 20:53:35,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-22 20:53:35,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-22 20:53:35,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-22 20:53:35,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-22 20:53:35,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-22 20:53:35,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-22 20:53:35,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-22 20:53:35,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-22 20:53:35,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-22 20:53:35,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-22 20:53:35,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-22 20:53:35,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-22 20:53:35,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-22 20:53:35,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-22 20:53:35,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-22 20:53:35,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-22 20:53:35,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-22 20:53:35,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-22 20:53:35,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-22 20:53:35,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-22 20:53:35,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-22 20:53:35,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-22 20:53:35,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-22 20:53:35,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-22 20:53:35,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-22 20:53:35,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-22 20:53:35,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-22 20:53:35,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-22 20:53:35,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-22 20:53:35,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-22 20:53:35,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-22 20:53:35,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-22 20:53:35,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-22 20:53:35,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-22 20:53:35,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-22 20:53:35,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-22 20:53:35,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-22 20:53:35,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-22 20:53:35,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-22 20:53:35,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-22 20:53:35,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-22 20:53:35,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-22 20:53:35,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-22 20:53:35,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-22 20:53:35,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-22 20:53:35,418 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021052254182399782); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03247030932489435, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021052254182399782); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:53:35,418 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-22 20:53:35,418 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013849234710906528, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021052254182399782); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03247030932489435, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021052254182399782); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-22 20:53:35,726 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 461 ms to try 5000 different iterations.  Found a solution that moves 960 regions; Going from a computed imbalance of 0.013849234710906528 to a new imbalance of 4.245109377881532E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005887925296510981); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006513389472789296); MoveCostFunction : (multiplier=7.0, imbalance=0.10666666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005887925296510981); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0023333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:53:35,728 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv698237866:81 , srv302582575:82 , srv966927625:82 , srv977431252:82 , srv1219084814:83 , srv1820981880:83 , srv1974638136:83 , srv337056998:83 , srv521953996:83 , srv686258804:83 , srv1079829720:84 , srv1088862020:84 , srv1129117392:84 , srv1180586844:84 , srv1470689115:84 , srv1559160519:84 , srv1592156699:84 , srv1620559906:84 , srv1757379976:84 , srv1797114143:84 , srv2143961996:84 , srv312633856:84 , srv389434929:84 , srv435607241:84 , srv787036813:84 , srv1025036698:85 , srv1232954078:85 , srv1300138830:85 , srv1396672047:85 , srv1576255176:85 , srv1659054370:85 , srv1800728599:85 , srv1830512933:85 , srv1847214032:85 , srv1920855422:85 , srv2012054449:85 , srv322229414:85 , srv347556955:85 , srv429970457:85 , srv563017216:85 , srv621635894:85 , srv638708047:85 , srv659958183:85 , srv676573771:85 , srv734837021:85 , srv79904751:85 , srv8440776:85 , srv100661390:86 , srv1156854385:86 , srv1188612978:86 , srv1228236412:86 , srv1239308738:86 , srv1310679201:86 , srv1541622808:86 , srv1542390325:86 , srv160346830:86 , srv1616695046:86 , srv1747343783:86 , srv1794410725:86 , srv1803830104:86 , srv1858042450:86 , srv192674803:86 , srv19407699:86 , srv1968457702:86 , srv1977884307:86 , srv2006894373:86 , srv2037649379:86 , srv250998093:86 , srv416909512:86 , srv478213630:86 , srv600073957:86 , srv606902129:86 , srv609134293:86 , srv613133173:86 , srv732762574:86 , srv887397105:86 , srv920190617:86 , srv983150188:86 , srv996060075:86 , srv1245391839:87 , srv1273180577:87 , srv1283402512:87 , srv1532980815:87 , srv1818163130:87 , srv1879907828:87 , srv1949472770:87 , srv2006311801:87 , srv2039896996:87 , srv2094228983:87 , srv609445283:87 , srv618546699:87 , srv646695554:87 , srv66550561:87 , srv730742580:87 , srv815214634:87 , srv996474162:87 , srv1060663830:88 , srv237825147:88 , srv380105146:89 , srv179883179:549 }
2024-04-22 20:53:35,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 20:53:35,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1025036698=1, srv237825147=57, srv996060075=98, srv19407699=45, srv1239308738=12, srv920190617=94, srv732762574=87, srv1800728599=35, srv1232954078=11, srv1300138830=16, srv1794410725=32, srv1879907828=42, srv996474162=99, srv1396672047=18, srv1974638136=48, srv2006894373=51, srv966927625=95, srv380105146=64, srv1156854385=6, srv1245391839=13, srv646695554=80, srv815214634=91, srv1129117392=5, srv1977884307=49, srv416909512=66, srv337056998=62, srv1470689115=19, srv8440776=92, srv977431252=96, srv609134293=74, srv1273180577=14, srv2143961996=56, srv1283402512=15, srv638708047=79, srv322229414=61, srv609445283=75, srv1228236412=10, srv621635894=78, srv734837021=88, srv563017216=71, srv1858042450=41, srv347556955=63, srv1219084814=9, srv600073957=72, srv787036813=89, srv686258804=84, srv1757379976=31, srv1847214032=40, srv2039896996=54, srv698237866=85, srv1616695046=27, srv659958183=81, srv1949472770=46, srv250998093=58, srv1803830104=36, srv160346830=26, srv1532980815=20, srv1620559906=28, srv730742580=86, srv435607241=68, srv389434929=65, srv100661390=0, srv1180586844=7, srv1818163130=37, srv66550561=82, srv1659054370=29, srv1747343783=30, srv606902129=73, srv983150188=97, srv1559160519=23, srv312633856=60, srv521953996=70, srv478213630=69, srv192674803=44, srv179883179=34, srv1541622808=21, srv302582575=59, srv1310679201=17, srv1968457702=47, srv618546699=77, srv1088862020=4, srv1542390325=22, srv1830512933=39, srv887397105=93, srv1576255176=24, srv1079829720=3, srv1920855422=43, srv1188612978=8, srv1820981880=38, srv2006311801=50, srv2012054449=52, srv2037649379=53, srv429970457=67, srv1592156699=25, srv2094228983=55, srv79904751=90, srv1797114143=33, srv676573771=83, srv1060663830=2, srv613133173=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 20:53:35,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 20:53:35,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 20:53:35,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 20:53:35,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 20:53:35,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-22 20:53:35,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-22 20:53:35,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-22 20:53:35,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-22 20:53:35,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-22 20:53:35,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-22 20:53:35,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-22 20:53:35,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-22 20:53:35,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-22 20:53:35,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-22 20:53:35,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-22 20:53:35,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-22 20:53:35,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-22 20:53:35,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-22 20:53:35,863 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005887925296510981); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006513389472789296); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005887925296510981); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0023333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:53:35,864 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-22 20:53:35,864 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.177888741146054E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005887925296510981); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006513389472789296); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005887925296510981); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0023333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-22 20:53:36,306 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 538 ms to try 5000 different iterations.  Found a solution that moves 201 regions; Going from a computed imbalance of 4.177888741146054E-4 to a new imbalance of 9.798154798867589E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.735366466207394E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.022333333333333334); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.735366466207394E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:53:36,307 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1273180577:83 , srv160346830:83 , srv19407699:83 , srv380105146:83 , srv600073957:83 , srv618546699:83 , srv676573771:83 , srv100661390:84 , srv1060663830:84 , srv1129117392:84 , srv1232954078:84 , srv1245391839:84 , srv1283402512:84 , srv1310679201:84 , srv1470689115:84 , srv1532980815:84 , srv1576255176:84 , srv1616695046:84 , srv1659054370:84 , srv1747343783:84 , srv1794410725:84 , srv1797114143:84 , srv1800728599:84 , srv1830512933:84 , srv1858042450:84 , srv192674803:84 , srv1974638136:84 , srv1977884307:84 , srv2006311801:84 , srv2012054449:84 , srv2037649379:84 , srv2039896996:84 , srv2094228983:84 , srv302582575:84 , srv322229414:84 , srv389434929:84 , srv429970457:84 , srv621635894:84 , srv638708047:84 , srv646695554:84 , srv686258804:84 , srv698237866:84 , srv730742580:84 , srv732762574:84 , srv996060075:84 , srv996474162:84 , srv1025036698:85 , srv1079829720:85 , srv1088862020:85 , srv1156854385:85 , srv1180586844:85 , srv1188612978:85 , srv1219084814:85 , srv1228236412:85 , srv1239308738:85 , srv1300138830:85 , srv1396672047:85 , srv1542390325:85 , srv1559160519:85 , srv1592156699:85 , srv1620559906:85 , srv1757379976:85 , srv1803830104:85 , srv1818163130:85 , srv1820981880:85 , srv1847214032:85 , srv1879907828:85 , srv1920855422:85 , srv1949472770:85 , srv2143961996:85 , srv237825147:85 , srv312633856:85 , srv337056998:85 , srv347556955:85 , srv478213630:85 , srv521953996:85 , srv563017216:85 , srv606902129:85 , srv609134293:85 , srv609445283:85 , srv613133173:85 , srv659958183:85 , srv66550561:85 , srv734837021:85 , srv787036813:85 , srv79904751:85 , srv815214634:85 , srv8440776:85 , srv887397105:85 , srv920190617:85 , srv966927625:85 , srv977431252:85 , srv983150188:85 , srv1541622808:86 , srv1968457702:86 , srv2006894373:86 , srv250998093:86 , srv416909512:86 , srv435607241:86 , srv179883179:632 }
2024-04-22 20:53:36,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 20:53:36,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1025036698=1, srv237825147=57, srv996060075=98, srv19407699=45, srv1239308738=12, srv920190617=94, srv732762574=87, srv1800728599=35, srv1232954078=11, srv1300138830=16, srv1794410725=32, srv1879907828=42, srv996474162=99, srv1396672047=18, srv1974638136=48, srv2006894373=51, srv966927625=95, srv380105146=64, srv1156854385=6, srv1245391839=13, srv646695554=80, srv815214634=91, srv1129117392=5, srv1977884307=49, srv416909512=66, srv337056998=62, srv1470689115=19, srv8440776=92, srv977431252=96, srv609134293=74, srv1273180577=14, srv2143961996=56, srv1283402512=15, srv638708047=79, srv322229414=61, srv609445283=75, srv1228236412=10, srv621635894=78, srv734837021=88, srv563017216=71, srv1858042450=41, srv347556955=63, srv1219084814=9, srv600073957=72, srv787036813=89, srv686258804=84, srv1757379976=31, srv1847214032=40, srv2039896996=54, srv698237866=85, srv1616695046=27, srv659958183=81, srv1949472770=46, srv250998093=58, srv1803830104=36, srv160346830=26, srv1532980815=20, srv1620559906=28, srv730742580=86, srv435607241=68, srv389434929=65, srv100661390=0, srv1180586844=7, srv1818163130=37, srv66550561=82, srv1659054370=29, srv1747343783=30, srv606902129=73, srv983150188=97, srv1559160519=23, srv312633856=60, srv521953996=70, srv478213630=69, srv192674803=44, srv179883179=34, srv1541622808=21, srv302582575=59, srv1310679201=17, srv1968457702=47, srv618546699=77, srv1088862020=4, srv1542390325=22, srv1830512933=39, srv887397105=93, srv1576255176=24, srv1079829720=3, srv1920855422=43, srv1188612978=8, srv1820981880=38, srv2006311801=50, srv2012054449=52, srv2037649379=53, srv429970457=67, srv1592156699=25, srv2094228983=55, srv79904751=90, srv1797114143=33, srv676573771=83, srv1060663830=2, srv613133173=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 20:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 20:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 20:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-22 20:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-22 20:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-22 20:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-22 20:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-22 20:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-22 20:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-22 20:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-22 20:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-22 20:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-22 20:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-22 20:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-22 20:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-22 20:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-22 20:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-22 20:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-22 20:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-22 20:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-22 20:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-22 20:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-22 20:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-22 20:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-22 20:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-22 20:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-22 20:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-22 20:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-22 20:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-22 20:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-22 20:53:36,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-22 20:53:36,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-22 20:53:36,459 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.390722717218522E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.735366466207394E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.735366466207394E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:53:36,459 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1273180577:83 , srv160346830:83 , srv19407699:83 , srv380105146:83 , srv600073957:83 , srv618546699:83 , srv676573771:83 , srv100661390:84 , srv1060663830:84 , srv1129117392:84 , srv1232954078:84 , srv1245391839:84 , srv1283402512:84 , srv1310679201:84 , srv1470689115:84 , srv1532980815:84 , srv1576255176:84 , srv1616695046:84 , srv1659054370:84 , srv1747343783:84 , srv1794410725:84 , srv1797114143:84 , srv1800728599:84 , srv1830512933:84 , srv1858042450:84 , srv192674803:84 , srv1974638136:84 , srv1977884307:84 , srv2006311801:84 , srv2012054449:84 , srv2037649379:84 , srv2039896996:84 , srv2094228983:84 , srv302582575:84 , srv322229414:84 , srv389434929:84 , srv429970457:84 , srv621635894:84 , srv638708047:84 , srv646695554:84 , srv686258804:84 , srv698237866:84 , srv730742580:84 , srv732762574:84 , srv996060075:84 , srv996474162:84 , srv1025036698:85 , srv1079829720:85 , srv1088862020:85 , srv1156854385:85 , srv1180586844:85 , srv1188612978:85 , srv1219084814:85 , srv1228236412:85 , srv1239308738:85 , srv1300138830:85 , srv1396672047:85 , srv1542390325:85 , srv1559160519:85 , srv1592156699:85 , srv1620559906:85 , srv1757379976:85 , srv1803830104:85 , srv1818163130:85 , srv1820981880:85 , srv1847214032:85 , srv1879907828:85 , srv1920855422:85 , srv1949472770:85 , srv2143961996:85 , srv237825147:85 , srv312633856:85 , srv337056998:85 , srv347556955:85 , srv478213630:85 , srv521953996:85 , srv563017216:85 , srv606902129:85 , srv609134293:85 , srv609445283:85 , srv613133173:85 , srv659958183:85 , srv66550561:85 , srv734837021:85 , srv787036813:85 , srv79904751:85 , srv815214634:85 , srv8440776:85 , srv887397105:85 , srv920190617:85 , srv966927625:85 , srv977431252:85 , srv983150188:85 , srv1541622808:86 , srv1968457702:86 , srv2006894373:86 , srv250998093:86 , srv416909512:86 , srv435607241:86 , srv179883179:632 }
2024-04-22 20:53:36,501 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=289 (was 289), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=363 (was 307) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=13537 (was 14058)
2024-04-22 20:53:36,516 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=289, MaxFileDescriptor=60000, SystemLoadAverage=363, ProcessCount=170, AvailableMemoryMB=13535
2024-04-22 20:53:36,517 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-22 20:53:36,517 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-04-22 20:53:36,529 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv10251332:84 , srv1129439889:84 , srv1221696662:84 , srv1274607166:84 , srv1333094660:84 , srv1353246661:84 , srv1355625870:84 , srv1372965086:84 , srv1531612503:84 , srv1551397736:84 , srv167866031:84 , srv1726131095:84 , srv1790004082:84 , srv1833079635:84 , srv2014257199:84 , srv2058629904:84 , srv2099343655:84 , srv226317955:84 , srv250125813:84 , srv262049518:84 , srv432658895:84 , srv437922674:84 , srv496229959:84 , srv527578433:84 , srv76293314:84 , srv822928096:84 , srv875195155:84 , srv912686670:84 , srv974382399:84 , srv1461941571:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-04-22 20:53:36,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 20:53:36,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv167866031=11, srv250125813=19, srv1531612503=9, srv226317955=18, srv822928096=26, srv2099343655=17, srv1726131095=12, srv527578433=24, srv10251332=0, srv1333094660=4, srv1461941571=8, srv1129439889=1, srv974382399=29, srv2014257199=15, srv76293314=25, srv1274607166=3, srv1353246661=5, srv1833079635=14, srv1221696662=2, srv1372965086=7, srv1355625870=6, srv262049518=20, srv1551397736=10, srv437922674=22, srv2058629904=16, srv432658895=21, srv496229959=23, srv875195155=27, srv1790004082=13, srv912686670=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 20:53:36,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 20:53:36,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 20:53:36,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 20:53:36,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 20:53:36,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 20:53:36,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 20:53:36,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 20:53:36,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 20:53:36,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 20:53:36,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 20:53:36,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 20:53:36,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 20:53:36,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 20:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 20:53:36,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-22 20:53:36,564 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1760963589489277, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:53:36,564 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912747917141252, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1760963589489277, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-04-22 20:55:36,551 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5744186 different iterations.  Found a solution that moves 1864 regions; Going from a computed imbalance of 0.9912747917141252 to a new imbalance of 4.3506689887128686E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6903703703703704, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:55:36,558 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv10251332:90 , srv1129439889:90 , srv1221696662:90 , srv1274607166:90 , srv1333094660:90 , srv1353246661:90 , srv1355625870:90 , srv1372965086:90 , srv1461941571:90 , srv1531612503:90 , srv1551397736:90 , srv167866031:90 , srv1726131095:90 , srv1790004082:90 , srv1833079635:90 , srv2014257199:90 , srv2058629904:90 , srv2099343655:90 , srv226317955:90 , srv250125813:90 , srv262049518:90 , srv432658895:90 , srv437922674:90 , srv496229959:90 , srv527578433:90 , srv76293314:90 , srv822928096:90 , srv875195155:90 , srv912686670:90 , srv974382399:90 }
2024-04-22 20:55:36,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 20:55:36,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv167866031=11, srv250125813=19, srv1531612503=9, srv226317955=18, srv822928096=26, srv2099343655=17, srv1726131095=12, srv527578433=24, srv10251332=0, srv1333094660=4, srv1461941571=8, srv1129439889=1, srv974382399=29, srv2014257199=15, srv76293314=25, srv1274607166=3, srv1353246661=5, srv1833079635=14, srv1221696662=2, srv1372965086=7, srv1355625870=6, srv262049518=20, srv1551397736=10, srv437922674=22, srv2058629904=16, srv432658895=21, srv496229959=23, srv875195155=27, srv1790004082=13, srv912686670=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 20:55:36,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 20:55:36,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 20:55:36,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 20:55:36,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 20:55:36,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 20:55:36,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-22 20:55:36,584 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 20:55:36,585 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv10251332:90 , srv1129439889:90 , srv1221696662:90 , srv1274607166:90 , srv1333094660:90 , srv1353246661:90 , srv1355625870:90 , srv1372965086:90 , srv1461941571:90 , srv1531612503:90 , srv1551397736:90 , srv167866031:90 , srv1726131095:90 , srv1790004082:90 , srv1833079635:90 , srv2014257199:90 , srv2058629904:90 , srv2099343655:90 , srv226317955:90 , srv250125813:90 , srv262049518:90 , srv432658895:90 , srv437922674:90 , srv496229959:90 , srv527578433:90 , srv76293314:90 , srv822928096:90 , srv875195155:90 , srv912686670:90 , srv974382399:90 }
2024-04-22 20:55:36,620 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=289 (was 289), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=237 (was 363), ProcessCount=170 (was 170), AvailableMemoryMB=13508 (was 13535)