2023-04-18 13:55:46,214 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-18 13:55:46,572 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-18 13:55:46,715 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-18 13:55:46,798 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-18 13:55:46,806 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-18 13:55:46,823 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=405, ProcessCount=170, AvailableMemoryMB=7742
2023-04-18 13:55:46,825 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-18 13:55:46,826 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-18 13:55:46,979 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1056653523:84 , srv1104495337:84 , srv1126097760:84 , srv1140956953:84 , srv1165316278:84 , srv1193819138:84 , srv1261001138:84 , srv1273085378:84 , srv1288526858:84 , srv1294437574:84 , srv1315148733:84 , srv1367462385:84 , srv1375260520:84 , srv1380511962:84 , srv1419531142:84 , srv1420109399:84 , srv1434001301:84 , srv144476102:84 , srv1453159330:84 , srv150478390:84 , srv1511901790:84 , srv1538728150:84 , srv1566970920:84 , srv1581719992:84 , srv1594984712:84 , srv1604150152:84 , srv1663175953:84 , srv1690745311:84 , srv1708536197:84 , srv1717016183:84 , srv1735726105:84 , srv1742204822:84 , srv1754753040:84 , srv1771061351:84 , srv1822254875:84 , srv1846351742:84 , srv1861271974:84 , srv1883593550:84 , srv190122813:84 , srv1926630449:84 , srv1931919704:84 , srv1935794430:84 , srv1950796305:84 , srv1968516906:84 , srv1997226302:84 , srv2021007065:84 , srv2026164648:84 , srv2049372744:84 , srv206698396:84 , srv2067439657:84 , srv2071205806:84 , srv2078297679:84 , srv2116053123:84 , srv2118091197:84 , srv2119064687:84 , srv2127008160:84 , srv237765794:84 , srv239868299:84 , srv250294205:84 , srv267334303:84 , srv27505135:84 , srv28718498:84 , srv313508775:84 , srv320538932:84 , srv320625574:84 , srv337910847:84 , srv348205051:84 , srv366268114:84 , srv40933158:84 , srv456622369:84 , srv463761171:84 , srv468595997:84 , srv502381062:84 , srv503558814:84 , srv514961543:84 , srv524124656:84 , srv557273590:84 , srv559578191:84 , srv588407959:84 , srv606864714:84 , srv628115602:84 , srv642956737:84 , srv652484785:84 , srv678659097:84 , srv686928200:84 , srv714036594:84 , srv747793089:84 , srv790295333:84 , srv823619988:84 , srv834726719:84 , srv837689264:84 , srv874016900:84 , srv8805934:84 , srv908808344:84 , srv934953704:84 , srv937709922:84 , srv938376329:84 , srv955753021:84 , srv976042877:84 , srv187257076:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-18 13:55:46,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-18 13:55:47,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1193819138=5, srv237765794=57, srv40933158=69, srv1056653523=0, srv1434001301=16, srv239868299=58, srv1511901790=20, srv834726719=90, srv1708536197=28, srv1883593550=38, srv27505135=61, srv1367462385=11, srv1717016183=29, srv2116053123=53, srv1273085378=7, srv955753021=98, srv348205051=67, srv747793089=87, srv678659097=84, srv1538728150=21, srv190122813=39, srv559578191=78, srv1140956953=3, srv1926630449=40, srv468595997=72, srv2026164648=47, srv652484785=83, srv1690745311=27, srv1581719992=23, srv790295333=88, srv1294437574=9, srv206698396=49, srv320538932=64, srv1663175953=26, srv714036594=86, srv908808344=94, srv2118091197=54, srv28718498=62, srv1935794430=42, srv1566970920=22, srv1594984712=24, srv320625574=65, srv1165316278=4, srv1754753040=32, srv1846351742=35, srv1261001138=6, srv1931919704=41, srv588407959=79, srv366268114=68, srv1380511962=13, srv144476102=17, srv686928200=85, srv1822254875=34, srv1771061351=33, srv250294205=59, srv1968516906=44, srv1375260520=12, srv2127008160=56, srv1126097760=2, srv2078297679=52, srv2021007065=46, srv2119064687=55, srv503558814=74, srv874016900=92, srv1950796305=43, srv2049372744=48, srv937709922=96, srv1604150152=25, srv514961543=75, srv1104495337=1, srv337910847=66, srv1420109399=15, srv502381062=73, srv187257076=37, srv1419531142=14, srv1742204822=31, srv8805934=93, srv313508775=63, srv823619988=89, srv837689264=91, srv1735726105=30, srv628115602=81, srv1861271974=36, srv557273590=77, srv1997226302=45, srv606864714=80, srv1315148733=10, srv2067439657=50, srv463761171=71, srv976042877=99, srv1453159330=18, srv934953704=95, srv2071205806=51, srv456622369=70, srv1288526858=8, srv938376329=97, srv150478390=19, srv642956737=82, srv524124656=76, srv267334303=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-18 13:55:47,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-18 13:55:47,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-18 13:55:47,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-18 13:55:47,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-18 13:55:47,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-18 13:55:47,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-18 13:55:47,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-18 13:55:47,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-18 13:55:47,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-18 13:55:47,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-18 13:55:47,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-18 13:55:47,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-18 13:55:47,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-18 13:55:47,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-18 13:55:47,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-18 13:55:47,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-18 13:55:47,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-18 13:55:47,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-18 13:55:47,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-18 13:55:47,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-18 13:55:47,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-18 13:55:47,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-18 13:55:47,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-18 13:55:47,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-18 13:55:47,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-18 13:55:47,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-18 13:55:47,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-18 13:55:47,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-18 13:55:47,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-18 13:55:47,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-18 13:55:47,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-18 13:55:47,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-18 13:55:47,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-18 13:55:47,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-18 13:55:47,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-18 13:55:47,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-18 13:55:47,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-18 13:55:47,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-18 13:55:47,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-18 13:55:47,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-18 13:55:47,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-18 13:55:47,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-18 13:55:47,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-18 13:55:47,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-18 13:55:47,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-18 13:55:47,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-18 13:55:47,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-18 13:55:47,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-18 13:55:47,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-18 13:55:47,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-18 13:55:47,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-18 13:55:47,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-18 13:55:47,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-18 13:55:47,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-18 13:55:47,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-18 13:55:47,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-18 13:55:47,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-18 13:55:47,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-18 13:55:47,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-18 13:55:47,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-18 13:55:47,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-18 13:55:47,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-18 13:55:47,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-18 13:55:47,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-18 13:55:47,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-18 13:55:47,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-18 13:55:47,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-18 13:55:47,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-18 13:55:47,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-18 13:55:47,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-18 13:55:47,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-18 13:55:47,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-18 13:55:47,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-18 13:55:47,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-18 13:55:47,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-18 13:55:47,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-18 13:55:47,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-18 13:55:47,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-18 13:55:47,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-18 13:55:47,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-18 13:55:47,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-18 13:55:47,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-18 13:55:47,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-18 13:55:47,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-18 13:55:47,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-18 13:55:47,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-18 13:55:47,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-18 13:55:47,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-18 13:55:47,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-18 13:55:47,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-18 13:55:47,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-18 13:55:47,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-18 13:55:47,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-18 13:55:47,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-18 13:55:47,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-18 13:55:47,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-18 13:55:47,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-18 13:55:47,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-18 13:55:47,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-18 13:55:47,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-18 13:55:47,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-18 13:55:47,405 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:47,405 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-18 13:55:47,406 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-18 13:55:48,417 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1213 ms to try 5000 different iterations.  Found a solution that moves 4691 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18220866119468668. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04060307318803905, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05430718628877596, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5212222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04060307318803905, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17725, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:48,436 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1419531142:80 , srv1663175953:80 , srv1140956953:81 , srv1375260520:81 , srv1420109399:81 , srv1581719992:81 , srv1883593550:81 , srv2071205806:81 , srv1434001301:82 , srv1926630449:82 , srv1968516906:82 , srv206698396:82 , srv468595997:82 , srv934953704:82 , srv1056653523:83 , srv1511901790:83 , srv2049372744:83 , srv2067439657:83 , srv250294205:83 , srv313508775:83 , srv1104495337:84 , srv1261001138:84 , srv1288526858:84 , srv1754753040:84 , srv1771061351:84 , srv1861271974:84 , srv1935794430:84 , srv366268114:84 , srv524124656:84 , srv678659097:84 , srv747793089:84 , srv955753021:84 , srv1126097760:85 , srv1165316278:85 , srv1273085378:85 , srv1315148733:85 , srv1367462385:85 , srv1594984712:85 , srv1708536197:85 , srv2119064687:85 , srv237765794:85 , srv40933158:85 , srv456622369:85 , srv628115602:85 , srv144476102:86 , srv150478390:86 , srv1538728150:86 , srv1566970920:86 , srv1950796305:86 , srv2078297679:86 , srv28718498:86 , srv348205051:86 , srv642956737:86 , srv652484785:86 , srv874016900:86 , srv8805934:86 , srv1453159330:87 , srv1931919704:87 , srv2026164648:87 , srv2118091197:87 , srv267334303:87 , srv320625574:87 , srv337910847:87 , srv502381062:87 , srv606864714:87 , srv714036594:87 , srv834726719:87 , srv937709922:87 , srv1690745311:88 , srv1822254875:88 , srv1997226302:88 , srv2021007065:88 , srv320538932:88 , srv559578191:88 , srv908808344:88 , srv1735726105:89 , srv1846351742:89 , srv2127008160:89 , srv463761171:89 , srv588407959:89 , srv790295333:89 , srv1294437574:90 , srv1742204822:90 , srv2116053123:90 , srv503558814:90 , srv938376329:90 , srv1604150152:91 , srv1717016183:91 , srv190122813:91 , srv239868299:92 , srv686928200:92 , srv823619988:92 , srv1380511962:93 , srv27505135:93 , srv837689264:93 , srv557273590:94 , srv514961543:95 , srv976042877:95 , srv1193819138:98 , srv187257076:450 }
2023-04-18 13:55:48,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-18 13:55:48,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1193819138=5, srv237765794=57, srv40933158=69, srv1056653523=0, srv1434001301=16, srv239868299=58, srv1511901790=20, srv834726719=90, srv1708536197=28, srv1883593550=38, srv27505135=61, srv1367462385=11, srv1717016183=29, srv2116053123=53, srv1273085378=7, srv955753021=98, srv348205051=67, srv747793089=87, srv678659097=84, srv1538728150=21, srv190122813=39, srv559578191=78, srv1140956953=3, srv1926630449=40, srv468595997=72, srv2026164648=47, srv652484785=83, srv1690745311=27, srv1581719992=23, srv790295333=88, srv1294437574=9, srv206698396=49, srv320538932=64, srv1663175953=26, srv714036594=86, srv908808344=94, srv2118091197=54, srv28718498=62, srv1935794430=42, srv1566970920=22, srv1594984712=24, srv320625574=65, srv1165316278=4, srv1754753040=32, srv1846351742=35, srv1261001138=6, srv1931919704=41, srv588407959=79, srv366268114=68, srv1380511962=13, srv144476102=17, srv686928200=85, srv1822254875=34, srv1771061351=33, srv250294205=59, srv1968516906=44, srv1375260520=12, srv2127008160=56, srv1126097760=2, srv2078297679=52, srv2021007065=46, srv2119064687=55, srv503558814=74, srv874016900=92, srv1950796305=43, srv2049372744=48, srv937709922=96, srv1604150152=25, srv514961543=75, srv1104495337=1, srv337910847=66, srv1420109399=15, srv502381062=73, srv187257076=37, srv1419531142=14, srv1742204822=31, srv8805934=93, srv313508775=63, srv823619988=89, srv837689264=91, srv1735726105=30, srv628115602=81, srv1861271974=36, srv557273590=77, srv1997226302=45, srv606864714=80, srv1315148733=10, srv2067439657=50, srv463761171=71, srv976042877=99, srv1453159330=18, srv934953704=95, srv2071205806=51, srv456622369=70, srv1288526858=8, srv938376329=97, srv150478390=19, srv642956737=82, srv524124656=76, srv267334303=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-18 13:55:48,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-18 13:55:48,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-18 13:55:48,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-18 13:55:48,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-18 13:55:48,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-18 13:55:48,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-18 13:55:48,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-18 13:55:48,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-18 13:55:48,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-18 13:55:48,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-18 13:55:48,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-18 13:55:48,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-18 13:55:48,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-18 13:55:48,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-18 13:55:48,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-18 13:55:48,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-18 13:55:48,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-18 13:55:48,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-18 13:55:48,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-18 13:55:48,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-18 13:55:48,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-18 13:55:48,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-18 13:55:48,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-18 13:55:48,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-18 13:55:48,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-18 13:55:48,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-18 13:55:48,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-18 13:55:48,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-18 13:55:48,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-18 13:55:48,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-18 13:55:48,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-18 13:55:48,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-18 13:55:48,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-18 13:55:48,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-18 13:55:48,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-18 13:55:48,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-18 13:55:48,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-18 13:55:48,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-18 13:55:48,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-18 13:55:48,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-18 13:55:48,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-18 13:55:48,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-18 13:55:48,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-18 13:55:48,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-18 13:55:48,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-18 13:55:48,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-18 13:55:48,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-18 13:55:48,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-18 13:55:48,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-18 13:55:48,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-18 13:55:48,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-18 13:55:48,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-18 13:55:48,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-18 13:55:48,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-18 13:55:48,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-18 13:55:48,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-18 13:55:48,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-18 13:55:48,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-18 13:55:48,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-18 13:55:48,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-18 13:55:48,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-18 13:55:48,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-18 13:55:48,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-18 13:55:48,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-18 13:55:48,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-18 13:55:48,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-18 13:55:48,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-18 13:55:48,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-18 13:55:48,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-18 13:55:48,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-18 13:55:48,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-18 13:55:48,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-18 13:55:48,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-18 13:55:48,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-18 13:55:48,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-18 13:55:48,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-18 13:55:48,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-18 13:55:48,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-18 13:55:48,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-18 13:55:48,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-18 13:55:48,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-18 13:55:48,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-18 13:55:48,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-18 13:55:48,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-18 13:55:48,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-18 13:55:48,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-18 13:55:48,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-18 13:55:48,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-18 13:55:48,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-18 13:55:48,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-18 13:55:48,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-18 13:55:48,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-18 13:55:48,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-18 13:55:48,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-18 13:55:48,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-18 13:55:48,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-18 13:55:48,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-18 13:55:48,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-18 13:55:48,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-18 13:55:48,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-18 13:55:48,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-18 13:55:48,698 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04060307318803905, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05430718628877596, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04060307318803905, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17725, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:48,698 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-18 13:55:48,699 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18217581411063186, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04060307318803905, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05430718628877596, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04060307318803905, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17725, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-18 13:55:49,049 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 494 ms to try 5000 different iterations.  Found a solution that moves 2662 regions; Going from a computed imbalance of 0.18217581411063186 to a new imbalance of 0.0180992396719829. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02197108371850503); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03150187495533336, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29577777777777775, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02197108371850503); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.015); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04808333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:49,063 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1193819138:75 , srv27505135:77 , srv514961543:77 , srv1380511962:79 , srv823619988:79 , srv557273590:80 , srv686928200:80 , srv1604150152:81 , srv1742204822:81 , srv2116053123:81 , srv503558814:81 , srv837689264:81 , srv1717016183:82 , srv1822254875:82 , srv190122813:82 , srv1997226302:82 , srv239868299:82 , srv463761171:82 , srv502381062:82 , srv938376329:82 , srv1846351742:83 , srv1950796305:83 , srv2021007065:83 , srv1294437574:84 , srv144476102:84 , srv1708536197:84 , srv1931919704:84 , srv2127008160:84 , srv320625574:84 , srv559578191:84 , srv588407959:84 , srv714036594:84 , srv834726719:84 , srv1165316278:85 , srv1690745311:85 , srv2026164648:85 , srv2078297679:85 , srv320538932:85 , srv348205051:85 , srv40933158:85 , srv456622369:85 , srv8805934:85 , srv908808344:85 , srv976042877:85 , srv1126097760:86 , srv1367462385:86 , srv1453159330:86 , srv1594984712:86 , srv2118091197:86 , srv2119064687:86 , srv237765794:86 , srv337910847:86 , srv524124656:86 , srv606864714:86 , srv642956737:86 , srv678659097:86 , srv790295333:86 , srv874016900:86 , srv955753021:86 , srv1273085378:87 , srv1315148733:87 , srv150478390:87 , srv1566970920:87 , srv1735726105:87 , srv1861271974:87 , srv1926630449:87 , srv652484785:87 , srv937709922:87 , srv1261001138:88 , srv1288526858:88 , srv1538728150:88 , srv1754753040:88 , srv1935794430:88 , srv250294205:88 , srv267334303:88 , srv313508775:88 , srv1056653523:89 , srv1104495337:89 , srv1375260520:89 , srv1581719992:89 , srv1771061351:89 , srv1968516906:89 , srv2067439657:89 , srv2071205806:89 , srv28718498:89 , srv366268114:89 , srv747793089:89 , srv1420109399:90 , srv1511901790:90 , srv1663175953:90 , srv206698396:90 , srv628115602:90 , srv934953704:90 , srv1140956953:91 , srv1434001301:91 , srv468595997:92 , srv1419531142:93 , srv1883593550:93 , srv2049372744:93 , srv187257076:519 }
2023-04-18 13:55:49,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-18 13:55:49,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1193819138=5, srv237765794=57, srv40933158=69, srv1056653523=0, srv1434001301=16, srv239868299=58, srv1511901790=20, srv834726719=90, srv1708536197=28, srv1883593550=38, srv27505135=61, srv1367462385=11, srv1717016183=29, srv2116053123=53, srv1273085378=7, srv955753021=98, srv348205051=67, srv747793089=87, srv678659097=84, srv1538728150=21, srv190122813=39, srv559578191=78, srv1140956953=3, srv1926630449=40, srv468595997=72, srv2026164648=47, srv652484785=83, srv1690745311=27, srv1581719992=23, srv790295333=88, srv1294437574=9, srv206698396=49, srv320538932=64, srv1663175953=26, srv714036594=86, srv908808344=94, srv2118091197=54, srv28718498=62, srv1935794430=42, srv1566970920=22, srv1594984712=24, srv320625574=65, srv1165316278=4, srv1754753040=32, srv1846351742=35, srv1261001138=6, srv1931919704=41, srv588407959=79, srv366268114=68, srv1380511962=13, srv144476102=17, srv686928200=85, srv1822254875=34, srv1771061351=33, srv250294205=59, srv1968516906=44, srv1375260520=12, srv2127008160=56, srv1126097760=2, srv2078297679=52, srv2021007065=46, srv2119064687=55, srv503558814=74, srv874016900=92, srv1950796305=43, srv2049372744=48, srv937709922=96, srv1604150152=25, srv514961543=75, srv1104495337=1, srv337910847=66, srv1420109399=15, srv502381062=73, srv187257076=37, srv1419531142=14, srv1742204822=31, srv8805934=93, srv313508775=63, srv823619988=89, srv837689264=91, srv1735726105=30, srv628115602=81, srv1861271974=36, srv557273590=77, srv1997226302=45, srv606864714=80, srv1315148733=10, srv2067439657=50, srv463761171=71, srv976042877=99, srv1453159330=18, srv934953704=95, srv2071205806=51, srv456622369=70, srv1288526858=8, srv938376329=97, srv150478390=19, srv642956737=82, srv524124656=76, srv267334303=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-18 13:55:49,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-18 13:55:49,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-18 13:55:49,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-18 13:55:49,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-18 13:55:49,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-18 13:55:49,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-18 13:55:49,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-18 13:55:49,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-18 13:55:49,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-18 13:55:49,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-18 13:55:49,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-18 13:55:49,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-18 13:55:49,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-18 13:55:49,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-18 13:55:49,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-18 13:55:49,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-18 13:55:49,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-18 13:55:49,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-18 13:55:49,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-18 13:55:49,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-18 13:55:49,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-18 13:55:49,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-18 13:55:49,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-18 13:55:49,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-18 13:55:49,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-18 13:55:49,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-18 13:55:49,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-18 13:55:49,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-18 13:55:49,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-18 13:55:49,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-18 13:55:49,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-18 13:55:49,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-18 13:55:49,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-18 13:55:49,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-18 13:55:49,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-18 13:55:49,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-18 13:55:49,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-18 13:55:49,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-18 13:55:49,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-18 13:55:49,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-18 13:55:49,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-18 13:55:49,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-18 13:55:49,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-18 13:55:49,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-18 13:55:49,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-18 13:55:49,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-18 13:55:49,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-18 13:55:49,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-18 13:55:49,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-18 13:55:49,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-18 13:55:49,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-18 13:55:49,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-18 13:55:49,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-18 13:55:49,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-18 13:55:49,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-18 13:55:49,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-18 13:55:49,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-18 13:55:49,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-18 13:55:49,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-18 13:55:49,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-18 13:55:49,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-18 13:55:49,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-18 13:55:49,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-18 13:55:49,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-18 13:55:49,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-18 13:55:49,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-18 13:55:49,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-18 13:55:49,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-18 13:55:49,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-18 13:55:49,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-18 13:55:49,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-18 13:55:49,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-18 13:55:49,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-18 13:55:49,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-18 13:55:49,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-18 13:55:49,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-18 13:55:49,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-18 13:55:49,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-18 13:55:49,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-18 13:55:49,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-18 13:55:49,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-18 13:55:49,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-18 13:55:49,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-18 13:55:49,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-18 13:55:49,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-18 13:55:49,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-18 13:55:49,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-18 13:55:49,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-18 13:55:49,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-18 13:55:49,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-18 13:55:49,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-18 13:55:49,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-18 13:55:49,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-18 13:55:49,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-18 13:55:49,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-18 13:55:49,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-18 13:55:49,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-18 13:55:49,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-18 13:55:49,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-18 13:55:49,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-18 13:55:49,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-18 13:55:49,237 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02197108371850503); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03150187495533336, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02197108371850503); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.015); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04808333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:49,237 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-18 13:55:49,238 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.018080599949588126, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02197108371850503); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03150187495533336, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02197108371850503); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.015); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04808333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-18 13:55:49,578 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 468 ms to try 5000 different iterations.  Found a solution that moves 1076 regions; Going from a computed imbalance of 0.018080599949588126 to a new imbalance of 5.087347611819454E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006104220173631566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006478835438717); MoveCostFunction : (multiplier=7.0, imbalance=0.11955555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006104220173631566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:49,581 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv976042877:78 , srv1735726105:81 , srv2049372744:82 , srv790295333:82 , srv1538728150:83 , srv1690745311:83 , srv190122813:83 , srv28718498:83 , srv628115602:83 , srv837689264:83 , srv1273085378:84 , srv1294437574:84 , srv1511901790:84 , srv1566970920:84 , srv1717016183:84 , srv1771061351:84 , srv1883593550:84 , srv2127008160:84 , srv239868299:84 , srv267334303:84 , srv468595997:84 , srv557273590:84 , srv652484785:84 , srv747793089:84 , srv937709922:84 , srv938376329:84 , srv1104495337:85 , srv1193819138:85 , srv1261001138:85 , srv1288526858:85 , srv1380511962:85 , srv1419531142:85 , srv1434001301:85 , srv1453159330:85 , srv150478390:85 , srv1604150152:85 , srv2026164648:85 , srv206698396:85 , srv2118091197:85 , srv320538932:85 , srv337910847:85 , srv366268114:85 , srv514961543:85 , srv588407959:85 , srv606864714:85 , srv686928200:85 , srv8805934:85 , srv908808344:85 , srv934953704:85 , srv1056653523:86 , srv1140956953:86 , srv1315148733:86 , srv1367462385:86 , srv1420109399:86 , srv1754753040:86 , srv1931919704:86 , srv1935794430:86 , srv2021007065:86 , srv2067439657:86 , srv2116053123:86 , srv2119064687:86 , srv237765794:86 , srv250294205:86 , srv313508775:86 , srv348205051:86 , srv463761171:86 , srv503558814:86 , srv559578191:86 , srv642956737:86 , srv714036594:86 , srv823619988:86 , srv874016900:86 , srv1126097760:87 , srv1375260520:87 , srv144476102:87 , srv1581719992:87 , srv1594984712:87 , srv1663175953:87 , srv1742204822:87 , srv1822254875:87 , srv1861271974:87 , srv1968516906:87 , srv1997226302:87 , srv2071205806:87 , srv2078297679:87 , srv320625574:87 , srv40933158:87 , srv456622369:87 , srv834726719:87 , srv955753021:87 , srv1165316278:88 , srv1846351742:88 , srv1926630449:88 , srv1950796305:88 , srv27505135:88 , srv502381062:88 , srv524124656:88 , srv678659097:88 , srv1708536197:89 , srv187257076:543 }
2023-04-18 13:55:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-18 13:55:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1193819138=5, srv237765794=57, srv40933158=69, srv1056653523=0, srv1434001301=16, srv239868299=58, srv1511901790=20, srv834726719=90, srv1708536197=28, srv1883593550=38, srv27505135=61, srv1367462385=11, srv1717016183=29, srv2116053123=53, srv1273085378=7, srv955753021=98, srv348205051=67, srv747793089=87, srv678659097=84, srv1538728150=21, srv190122813=39, srv559578191=78, srv1140956953=3, srv1926630449=40, srv468595997=72, srv2026164648=47, srv652484785=83, srv1690745311=27, srv1581719992=23, srv790295333=88, srv1294437574=9, srv206698396=49, srv320538932=64, srv1663175953=26, srv714036594=86, srv908808344=94, srv2118091197=54, srv28718498=62, srv1935794430=42, srv1566970920=22, srv1594984712=24, srv320625574=65, srv1165316278=4, srv1754753040=32, srv1846351742=35, srv1261001138=6, srv1931919704=41, srv588407959=79, srv366268114=68, srv1380511962=13, srv144476102=17, srv686928200=85, srv1822254875=34, srv1771061351=33, srv250294205=59, srv1968516906=44, srv1375260520=12, srv2127008160=56, srv1126097760=2, srv2078297679=52, srv2021007065=46, srv2119064687=55, srv503558814=74, srv874016900=92, srv1950796305=43, srv2049372744=48, srv937709922=96, srv1604150152=25, srv514961543=75, srv1104495337=1, srv337910847=66, srv1420109399=15, srv502381062=73, srv187257076=37, srv1419531142=14, srv1742204822=31, srv8805934=93, srv313508775=63, srv823619988=89, srv837689264=91, srv1735726105=30, srv628115602=81, srv1861271974=36, srv557273590=77, srv1997226302=45, srv606864714=80, srv1315148733=10, srv2067439657=50, srv463761171=71, srv976042877=99, srv1453159330=18, srv934953704=95, srv2071205806=51, srv456622369=70, srv1288526858=8, srv938376329=97, srv150478390=19, srv642956737=82, srv524124656=76, srv267334303=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-18 13:55:49,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-18 13:55:49,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-18 13:55:49,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-18 13:55:49,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-18 13:55:49,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-18 13:55:49,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-18 13:55:49,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-18 13:55:49,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-18 13:55:49,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-18 13:55:49,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-18 13:55:49,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-18 13:55:49,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-18 13:55:49,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-18 13:55:49,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-18 13:55:49,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-18 13:55:49,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-18 13:55:49,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-18 13:55:49,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-18 13:55:49,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-18 13:55:49,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-18 13:55:49,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-18 13:55:49,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-18 13:55:49,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-18 13:55:49,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-18 13:55:49,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-18 13:55:49,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-18 13:55:49,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-18 13:55:49,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-18 13:55:49,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-18 13:55:49,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-18 13:55:49,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-18 13:55:49,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-18 13:55:49,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-18 13:55:49,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-18 13:55:49,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-18 13:55:49,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-18 13:55:49,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-18 13:55:49,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-18 13:55:49,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-18 13:55:49,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-18 13:55:49,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-18 13:55:49,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-18 13:55:49,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-18 13:55:49,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-18 13:55:49,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-18 13:55:49,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-18 13:55:49,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-18 13:55:49,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-18 13:55:49,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-18 13:55:49,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-18 13:55:49,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-18 13:55:49,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-18 13:55:49,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-18 13:55:49,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-18 13:55:49,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-18 13:55:49,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-18 13:55:49,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-18 13:55:49,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-18 13:55:49,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-18 13:55:49,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-18 13:55:49,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-18 13:55:49,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-18 13:55:49,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-18 13:55:49,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-18 13:55:49,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-18 13:55:49,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-18 13:55:49,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-18 13:55:49,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-18 13:55:49,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-18 13:55:49,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-18 13:55:49,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-18 13:55:49,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-18 13:55:49,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-18 13:55:49,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-18 13:55:49,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-18 13:55:49,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-18 13:55:49,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-18 13:55:49,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-18 13:55:49,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-18 13:55:49,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-18 13:55:49,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-18 13:55:49,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-18 13:55:49,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-18 13:55:49,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-18 13:55:49,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-18 13:55:49,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-18 13:55:49,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-18 13:55:49,751 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006104220173631566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006478835438717); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006104220173631566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:49,751 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-18 13:55:49,752 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.012004481478439E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006104220173631566); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006478835438717); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006104220173631566); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-18 13:55:50,102 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 470 ms to try 5000 different iterations.  Found a solution that moves 215 regions; Going from a computed imbalance of 5.012004481478439E-4 to a new imbalance of 2.6824557645782598E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.02388888888888889); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:50,103 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1846351742:82 , srv267334303:83 , srv320625574:83 , srv1056653523:84 , srv1104495337:84 , srv1126097760:84 , srv1165316278:84 , srv1193819138:84 , srv1294437574:84 , srv1315148733:84 , srv1419531142:84 , srv1434001301:84 , srv1453159330:84 , srv150478390:84 , srv1594984712:84 , srv1708536197:84 , srv1754753040:84 , srv1861271974:84 , srv1883593550:84 , srv1926630449:84 , srv1935794430:84 , srv1968516906:84 , srv206698396:84 , srv2067439657:84 , srv2078297679:84 , srv2118091197:84 , srv239868299:84 , srv27505135:84 , srv28718498:84 , srv320538932:84 , srv337910847:84 , srv366268114:84 , srv468595997:84 , srv524124656:84 , srv557273590:84 , srv559578191:84 , srv588407959:84 , srv606864714:84 , srv628115602:84 , srv642956737:84 , srv678659097:84 , srv834726719:84 , srv874016900:84 , srv908808344:84 , srv937709922:84 , srv976042877:84 , srv1140956953:85 , srv1261001138:85 , srv1288526858:85 , srv1367462385:85 , srv1375260520:85 , srv1380511962:85 , srv1420109399:85 , srv144476102:85 , srv1511901790:85 , srv1538728150:85 , srv1566970920:85 , srv1581719992:85 , srv1604150152:85 , srv1663175953:85 , srv1690745311:85 , srv1717016183:85 , srv1735726105:85 , srv1742204822:85 , srv1771061351:85 , srv1822254875:85 , srv1931919704:85 , srv1950796305:85 , srv1997226302:85 , srv2021007065:85 , srv2026164648:85 , srv2049372744:85 , srv2071205806:85 , srv2116053123:85 , srv2119064687:85 , srv237765794:85 , srv250294205:85 , srv313508775:85 , srv348205051:85 , srv40933158:85 , srv456622369:85 , srv463761171:85 , srv502381062:85 , srv503558814:85 , srv514961543:85 , srv652484785:85 , srv686928200:85 , srv714036594:85 , srv747793089:85 , srv790295333:85 , srv823619988:85 , srv837689264:85 , srv934953704:85 , srv955753021:85 , srv1273085378:86 , srv190122813:86 , srv2127008160:86 , srv8805934:86 , srv938376329:86 , srv187257076:630 }
2023-04-18 13:55:50,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-18 13:55:50,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1193819138=5, srv237765794=57, srv40933158=69, srv1056653523=0, srv1434001301=16, srv239868299=58, srv1511901790=20, srv834726719=90, srv1708536197=28, srv1883593550=38, srv27505135=61, srv1367462385=11, srv1717016183=29, srv2116053123=53, srv1273085378=7, srv955753021=98, srv348205051=67, srv747793089=87, srv678659097=84, srv1538728150=21, srv190122813=39, srv559578191=78, srv1140956953=3, srv1926630449=40, srv468595997=72, srv2026164648=47, srv652484785=83, srv1690745311=27, srv1581719992=23, srv790295333=88, srv1294437574=9, srv206698396=49, srv320538932=64, srv1663175953=26, srv714036594=86, srv908808344=94, srv2118091197=54, srv28718498=62, srv1935794430=42, srv1566970920=22, srv1594984712=24, srv320625574=65, srv1165316278=4, srv1754753040=32, srv1846351742=35, srv1261001138=6, srv1931919704=41, srv588407959=79, srv366268114=68, srv1380511962=13, srv144476102=17, srv686928200=85, srv1822254875=34, srv1771061351=33, srv250294205=59, srv1968516906=44, srv1375260520=12, srv2127008160=56, srv1126097760=2, srv2078297679=52, srv2021007065=46, srv2119064687=55, srv503558814=74, srv874016900=92, srv1950796305=43, srv2049372744=48, srv937709922=96, srv1604150152=25, srv514961543=75, srv1104495337=1, srv337910847=66, srv1420109399=15, srv502381062=73, srv187257076=37, srv1419531142=14, srv1742204822=31, srv8805934=93, srv313508775=63, srv823619988=89, srv837689264=91, srv1735726105=30, srv628115602=81, srv1861271974=36, srv557273590=77, srv1997226302=45, srv606864714=80, srv1315148733=10, srv2067439657=50, srv463761171=71, srv976042877=99, srv1453159330=18, srv934953704=95, srv2071205806=51, srv456622369=70, srv1288526858=8, srv938376329=97, srv150478390=19, srv642956737=82, srv524124656=76, srv267334303=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-18 13:55:50,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-18 13:55:50,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-18 13:55:50,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-18 13:55:50,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-18 13:55:50,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-18 13:55:50,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-18 13:55:50,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-18 13:55:50,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-18 13:55:50,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-18 13:55:50,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-18 13:55:50,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-18 13:55:50,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-18 13:55:50,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-18 13:55:50,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-18 13:55:50,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-18 13:55:50,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-18 13:55:50,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-18 13:55:50,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-18 13:55:50,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-18 13:55:50,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-18 13:55:50,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-18 13:55:50,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-18 13:55:50,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-18 13:55:50,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-18 13:55:50,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-18 13:55:50,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-18 13:55:50,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-18 13:55:50,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-18 13:55:50,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-18 13:55:50,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-18 13:55:50,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-18 13:55:50,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-18 13:55:50,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-18 13:55:50,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-18 13:55:50,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-18 13:55:50,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-18 13:55:50,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-18 13:55:50,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-18 13:55:50,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-18 13:55:50,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-18 13:55:50,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-18 13:55:50,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-18 13:55:50,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-18 13:55:50,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-18 13:55:50,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-18 13:55:50,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-18 13:55:50,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-18 13:55:50,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-18 13:55:50,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-18 13:55:50,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-18 13:55:50,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-18 13:55:50,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-18 13:55:50,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-18 13:55:50,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-18 13:55:50,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-18 13:55:50,247 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:50,248 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-18 13:55:50,248 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.5319095468894294E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-18 13:55:50,501 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 358 ms to try 5000 different iterations.  Found a solution that moves 28 regions; Going from a computed imbalance of 2.5319095468894294E-5 to a new imbalance of 6.2281389302068456E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.003111111111111111); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:50,501 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1140956953:83 , srv1742204822:83 , srv2049372744:83 , srv2127008160:83 , srv1056653523:84 , srv1104495337:84 , srv1126097760:84 , srv1165316278:84 , srv1193819138:84 , srv1261001138:84 , srv1273085378:84 , srv1288526858:84 , srv1294437574:84 , srv1315148733:84 , srv1367462385:84 , srv1375260520:84 , srv1380511962:84 , srv1419531142:84 , srv1420109399:84 , srv1434001301:84 , srv144476102:84 , srv1453159330:84 , srv150478390:84 , srv1511901790:84 , srv1538728150:84 , srv1566970920:84 , srv1581719992:84 , srv1594984712:84 , srv1604150152:84 , srv1663175953:84 , srv1708536197:84 , srv1717016183:84 , srv1735726105:84 , srv1754753040:84 , srv1771061351:84 , srv1822254875:84 , srv1846351742:84 , srv1861271974:84 , srv1883593550:84 , srv190122813:84 , srv1931919704:84 , srv1935794430:84 , srv1950796305:84 , srv1968516906:84 , srv1997226302:84 , srv2021007065:84 , srv2026164648:84 , srv2067439657:84 , srv2071205806:84 , srv2078297679:84 , srv2116053123:84 , srv2118091197:84 , srv2119064687:84 , srv237765794:84 , srv239868299:84 , srv250294205:84 , srv267334303:84 , srv27505135:84 , srv28718498:84 , srv313508775:84 , srv320538932:84 , srv337910847:84 , srv348205051:84 , srv366268114:84 , srv40933158:84 , srv456622369:84 , srv463761171:84 , srv468595997:84 , srv502381062:84 , srv503558814:84 , srv514961543:84 , srv524124656:84 , srv557273590:84 , srv559578191:84 , srv588407959:84 , srv606864714:84 , srv628115602:84 , srv642956737:84 , srv652484785:84 , srv678659097:84 , srv686928200:84 , srv714036594:84 , srv747793089:84 , srv790295333:84 , srv823619988:84 , srv834726719:84 , srv837689264:84 , srv874016900:84 , srv8805934:84 , srv908808344:84 , srv934953704:84 , srv937709922:84 , srv938376329:84 , srv955753021:84 , srv976042877:84 , srv1690745311:85 , srv1926630449:85 , srv206698396:85 , srv320625574:85 , srv187257076:684 }
2023-04-18 13:55:50,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-18 13:55:50,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1193819138=5, srv237765794=57, srv40933158=69, srv1056653523=0, srv1434001301=16, srv239868299=58, srv1511901790=20, srv834726719=90, srv1708536197=28, srv1883593550=38, srv27505135=61, srv1367462385=11, srv1717016183=29, srv2116053123=53, srv1273085378=7, srv955753021=98, srv348205051=67, srv747793089=87, srv678659097=84, srv1538728150=21, srv190122813=39, srv559578191=78, srv1140956953=3, srv1926630449=40, srv468595997=72, srv2026164648=47, srv652484785=83, srv1690745311=27, srv1581719992=23, srv790295333=88, srv1294437574=9, srv206698396=49, srv320538932=64, srv1663175953=26, srv714036594=86, srv908808344=94, srv2118091197=54, srv28718498=62, srv1935794430=42, srv1566970920=22, srv1594984712=24, srv320625574=65, srv1165316278=4, srv1754753040=32, srv1846351742=35, srv1261001138=6, srv1931919704=41, srv588407959=79, srv366268114=68, srv1380511962=13, srv144476102=17, srv686928200=85, srv1822254875=34, srv1771061351=33, srv250294205=59, srv1968516906=44, srv1375260520=12, srv2127008160=56, srv1126097760=2, srv2078297679=52, srv2021007065=46, srv2119064687=55, srv503558814=74, srv874016900=92, srv1950796305=43, srv2049372744=48, srv937709922=96, srv1604150152=25, srv514961543=75, srv1104495337=1, srv337910847=66, srv1420109399=15, srv502381062=73, srv187257076=37, srv1419531142=14, srv1742204822=31, srv8805934=93, srv313508775=63, srv823619988=89, srv837689264=91, srv1735726105=30, srv628115602=81, srv1861271974=36, srv557273590=77, srv1997226302=45, srv606864714=80, srv1315148733=10, srv2067439657=50, srv463761171=71, srv976042877=99, srv1453159330=18, srv934953704=95, srv2071205806=51, srv456622369=70, srv1288526858=8, srv938376329=97, srv150478390=19, srv642956737=82, srv524124656=76, srv267334303=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-18 13:55:50,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-18 13:55:50,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-18 13:55:50,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-18 13:55:50,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-18 13:55:50,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-18 13:55:50,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-18 13:55:50,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-18 13:55:50,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-18 13:55:50,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-18 13:55:50,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-18 13:55:50,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-18 13:55:50,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-18 13:55:50,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-18 13:55:50,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-18 13:55:50,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-18 13:55:50,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-18 13:55:50,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-18 13:55:50,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-18 13:55:50,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-18 13:55:50,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-18 13:55:50,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-18 13:55:50,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-18 13:55:50,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-18 13:55:50,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-18 13:55:50,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-18 13:55:50,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-18 13:55:50,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-18 13:55:50,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-18 13:55:50,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-18 13:55:50,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-18 13:55:50,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-18 13:55:50,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-18 13:55:50,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-18 13:55:50,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-18 13:55:50,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-18 13:55:50,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-18 13:55:50,683 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.032078739728369E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:50,683 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1140956953:83 , srv1742204822:83 , srv2049372744:83 , srv2127008160:83 , srv1056653523:84 , srv1104495337:84 , srv1126097760:84 , srv1165316278:84 , srv1193819138:84 , srv1261001138:84 , srv1273085378:84 , srv1288526858:84 , srv1294437574:84 , srv1315148733:84 , srv1367462385:84 , srv1375260520:84 , srv1380511962:84 , srv1419531142:84 , srv1420109399:84 , srv1434001301:84 , srv144476102:84 , srv1453159330:84 , srv150478390:84 , srv1511901790:84 , srv1538728150:84 , srv1566970920:84 , srv1581719992:84 , srv1594984712:84 , srv1604150152:84 , srv1663175953:84 , srv1708536197:84 , srv1717016183:84 , srv1735726105:84 , srv1754753040:84 , srv1771061351:84 , srv1822254875:84 , srv1846351742:84 , srv1861271974:84 , srv1883593550:84 , srv190122813:84 , srv1931919704:84 , srv1935794430:84 , srv1950796305:84 , srv1968516906:84 , srv1997226302:84 , srv2021007065:84 , srv2026164648:84 , srv2067439657:84 , srv2071205806:84 , srv2078297679:84 , srv2116053123:84 , srv2118091197:84 , srv2119064687:84 , srv237765794:84 , srv239868299:84 , srv250294205:84 , srv267334303:84 , srv27505135:84 , srv28718498:84 , srv313508775:84 , srv320538932:84 , srv337910847:84 , srv348205051:84 , srv366268114:84 , srv40933158:84 , srv456622369:84 , srv463761171:84 , srv468595997:84 , srv502381062:84 , srv503558814:84 , srv514961543:84 , srv524124656:84 , srv557273590:84 , srv559578191:84 , srv588407959:84 , srv606864714:84 , srv628115602:84 , srv642956737:84 , srv652484785:84 , srv678659097:84 , srv686928200:84 , srv714036594:84 , srv747793089:84 , srv790295333:84 , srv823619988:84 , srv834726719:84 , srv837689264:84 , srv874016900:84 , srv8805934:84 , srv908808344:84 , srv934953704:84 , srv937709922:84 , srv938376329:84 , srv955753021:84 , srv976042877:84 , srv1690745311:85 , srv1926630449:85 , srv206698396:85 , srv320625574:85 , srv187257076:684 }
2023-04-18 13:55:50,722 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=453 (was 405) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 170), AvailableMemoryMB=7269 (was 7742)
2023-04-18 13:55:50,733 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=453, ProcessCount=169, AvailableMemoryMB=7269
2023-04-18 13:55:50,734 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-18 13:55:50,735 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-18 13:55:50,751 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1006354607:84 , srv1156024317:84 , srv1199055767:84 , srv1208427820:84 , srv1298963128:84 , srv130946846:84 , srv1376896560:84 , srv1433425355:84 , srv1438634521:84 , srv1462168876:84 , srv1512677579:84 , srv1639819940:84 , srv1681102365:84 , srv1689750898:84 , srv1815294002:84 , srv281168989:84 , srv338221701:84 , srv386017044:84 , srv464844600:84 , srv494012951:84 , srv503351305:84 , srv537772045:84 , srv636639798:84 , srv654741916:84 , srv682572715:84 , srv75139045:84 , srv896689278:84 , srv931736125:84 , srv964143754:84 , srv1468743234:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-18 13:55:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-18 13:55:50,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv75139045=26, srv1199055767=2, srv1208427820=3, srv1639819940=12, srv1512677579=11, srv130946846=5, srv1433425355=7, srv682572715=25, srv537772045=22, srv636639798=23, srv964143754=29, srv1298963128=4, srv281168989=16, srv386017044=18, srv1006354607=0, srv1438634521=8, srv1689750898=14, srv494012951=20, srv503351305=21, srv1462168876=9, srv1156024317=1, srv338221701=17, srv931736125=28, srv464844600=19, srv654741916=24, srv1681102365=13, srv896689278=27, srv1468743234=10, srv1376896560=6, srv1815294002=15} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-18 13:55:50,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-18 13:55:50,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-18 13:55:50,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-18 13:55:50,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-18 13:55:50,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-18 13:55:50,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-18 13:55:50,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-18 13:55:50,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-18 13:55:50,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-18 13:55:50,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-18 13:55:50,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-18 13:55:50,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-18 13:55:50,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-18 13:55:50,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-18 13:55:50,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-18 13:55:50,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-18 13:55:50,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-18 13:55:50,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-18 13:55:50,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-18 13:55:50,806 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0494476414912381, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:55:50,806 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912348851168006, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0494476414912381, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-18 13:57:50,778 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6163280 different iterations.  Found a solution that moves 1845 regions; Going from a computed imbalance of 0.9912348851168006 to a new imbalance of 4.306322040866546E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6833333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:57:50,784 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1006354607:90 , srv1156024317:90 , srv1199055767:90 , srv1208427820:90 , srv1298963128:90 , srv130946846:90 , srv1376896560:90 , srv1433425355:90 , srv1438634521:90 , srv1462168876:90 , srv1468743234:90 , srv1512677579:90 , srv1639819940:90 , srv1681102365:90 , srv1689750898:90 , srv1815294002:90 , srv281168989:90 , srv338221701:90 , srv386017044:90 , srv464844600:90 , srv494012951:90 , srv503351305:90 , srv537772045:90 , srv636639798:90 , srv654741916:90 , srv682572715:90 , srv75139045:90 , srv896689278:90 , srv931736125:90 , srv964143754:90 }
2023-04-18 13:57:50,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-18 13:57:50,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv75139045=26, srv1199055767=2, srv1208427820=3, srv1639819940=12, srv1512677579=11, srv130946846=5, srv1433425355=7, srv682572715=25, srv537772045=22, srv636639798=23, srv964143754=29, srv1298963128=4, srv281168989=16, srv386017044=18, srv1006354607=0, srv1438634521=8, srv1689750898=14, srv494012951=20, srv503351305=21, srv1462168876=9, srv1156024317=1, srv338221701=17, srv931736125=28, srv464844600=19, srv654741916=24, srv1681102365=13, srv896689278=27, srv1468743234=10, srv1376896560=6, srv1815294002=15} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-18 13:57:50,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-18 13:57:50,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-18 13:57:50,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-18 13:57:50,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-18 13:57:50,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-18 13:57:50,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-18 13:57:50,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-18 13:57:50,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-18 13:57:50,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-18 13:57:50,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-18 13:57:50,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-18 13:57:50,802 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-18 13:57:50,802 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1006354607:90 , srv1156024317:90 , srv1199055767:90 , srv1208427820:90 , srv1298963128:90 , srv130946846:90 , srv1376896560:90 , srv1433425355:90 , srv1438634521:90 , srv1462168876:90 , srv1468743234:90 , srv1512677579:90 , srv1639819940:90 , srv1681102365:90 , srv1689750898:90 , srv1815294002:90 , srv281168989:90 , srv338221701:90 , srv386017044:90 , srv464844600:90 , srv494012951:90 , srv503351305:90 , srv537772045:90 , srv636639798:90 , srv654741916:90 , srv682572715:90 , srv75139045:90 , srv896689278:90 , srv931736125:90 , srv964143754:90 }
2023-04-18 13:57:50,814 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 453), ProcessCount=173 (was 169) - ProcessCount LEAK? -, AvailableMemoryMB=7261 (was 7269)