2023-08-21 09:53:11,427 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-21 09:53:11,759 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-21 09:53:11,883 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-21 09:53:11,949 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-21 09:53:11,956 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-21 09:53:11,969 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=338, ProcessCount=172, AvailableMemoryMB=11354
2023-08-21 09:53:11,971 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-21 09:53:11,972 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-21 09:53:12,122 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1028453383:84 , srv1037298215:84 , srv1069741736:84 , srv1086743529:84 , srv1093717479:84 , srv1119901530:84 , srv1128932182:84 , srv1143810146:84 , srv1152975546:84 , srv1157042046:84 , srv1173836134:84 , srv1179113431:84 , srv1182100125:84 , srv1211366566:84 , srv1217752864:84 , srv122005604:84 , srv1243744647:84 , srv1264388015:84 , srv1268025172:84 , srv1269939408:84 , srv1280271603:84 , srv1291137582:84 , srv1300994718:84 , srv1312770021:84 , srv1324166898:84 , srv1347678568:84 , srv1353891104:84 , srv1358309262:84 , srv1362828573:84 , srv1397558207:84 , srv1406409015:84 , srv1432747485:84 , srv1445393742:84 , srv1456173527:84 , srv148346263:84 , srv1487842283:84 , srv1490606202:84 , srv1505584559:84 , srv1541899612:84 , srv1548016313:84 , srv1566076759:84 , srv1571849364:84 , srv1585373032:84 , srv1595926573:84 , srv1641011539:84 , srv1678504986:84 , srv1704774236:84 , srv1730179451:84 , srv1739182201:84 , srv1758753011:84 , srv1766579067:84 , srv1860730761:84 , srv1866788701:84 , srv1900800830:84 , srv1911661909:84 , srv19232699:84 , srv1949041295:84 , srv196943017:84 , srv198846756:84 , srv2006298590:84 , srv2028395886:84 , srv2066731760:84 , srv2077435605:84 , srv2099679155:84 , srv2121748910:84 , srv2124143145:84 , srv22761114:84 , srv26654097:84 , srv29595577:84 , srv319470219:84 , srv342223849:84 , srv346933771:84 , srv351415251:84 , srv359959197:84 , srv378474511:84 , srv393777087:84 , srv420522325:84 , srv440873943:84 , srv454110508:84 , srv478383151:84 , srv580501338:84 , srv585560135:84 , srv607308238:84 , srv622750390:84 , srv643580617:84 , srv650283674:84 , srv740429402:84 , srv76535791:84 , srv771251780:84 , srv771375226:84 , srv772430203:84 , srv788903913:84 , srv818626894:84 , srv844023907:84 , srv845204158:84 , srv883698291:84 , srv914272400:84 , srv990545135:84 , srv995309455:84 , srv1219454509:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-21 09:53:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-21 09:53:12,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv22761114=67, srv2121748910=65, srv342223849=71, srv1211366566=13, srv643580617=85, srv1678504986=46, srv1766579067=51, srv772430203=91, srv378474511=75, srv2124143145=66, srv1219454509=15, srv1490606202=37, srv771251780=89, srv1028453383=0, srv1739182201=49, srv1758753011=50, srv844023907=94, srv622750390=84, srv76535791=88, srv2066731760=62, srv1143810146=7, srv393777087=76, srv1173836134=10, srv1037298215=1, srv1397558207=30, srv1866788701=53, srv1243744647=17, srv1353891104=27, srv1641011539=45, srv454110508=79, srv883698291=96, srv1406409015=31, srv1280271603=21, srv148346263=35, srv995309455=99, srv1217752864=14, srv1362828573=29, srv359959197=74, srv1291137582=22, srv1445393742=33, srv990545135=98, srv1595926573=44, srv788903913=92, srv1182100125=12, srv1119901530=5, srv2006298590=60, srv1566076759=41, srv650283674=86, srv580501338=81, srv1911661909=55, srv1268025172=19, srv440873943=78, srv914272400=97, srv585560135=82, srv198846756=59, srv1432747485=32, srv122005604=16, srv1300994718=23, srv1548016313=40, srv1571849364=42, srv1324166898=25, srv1541899612=39, srv1093717479=4, srv26654097=68, srv1860730761=52, srv319470219=70, srv2028395886=61, srv1949041295=57, srv2099679155=64, srv2077435605=63, srv1505584559=38, srv771375226=90, srv1704774236=47, srv1128932182=6, srv1086743529=3, srv607308238=83, srv1487842283=36, srv29595577=69, srv1312770021=24, srv740429402=87, srv1157042046=9, srv1456173527=34, srv478383151=80, srv818626894=93, srv1179113431=11, srv1347678568=26, srv196943017=58, srv1585373032=43, srv19232699=56, srv1264388015=18, srv1269939408=20, srv1900800830=54, srv420522325=77, srv1069741736=2, srv845204158=95, srv346933771=72, srv351415251=73, srv1152975546=8, srv1358309262=28, srv1730179451=48} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-21 09:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-21 09:53:12,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-21 09:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-21 09:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-21 09:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-21 09:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-21 09:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-21 09:53:12,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-21 09:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-21 09:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-21 09:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-21 09:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-21 09:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-21 09:53:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-21 09:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-21 09:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-21 09:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-21 09:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-21 09:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-21 09:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-21 09:53:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-21 09:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-21 09:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-21 09:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-21 09:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-21 09:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-21 09:53:12,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-21 09:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-21 09:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-21 09:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-21 09:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-21 09:53:12,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-21 09:53:12,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-21 09:53:12,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-21 09:53:12,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-21 09:53:12,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-21 09:53:12,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-21 09:53:12,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-21 09:53:12,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-21 09:53:12,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-21 09:53:12,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-21 09:53:12,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-21 09:53:12,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-21 09:53:12,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-21 09:53:12,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-21 09:53:12,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-21 09:53:12,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-21 09:53:12,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-21 09:53:12,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-21 09:53:12,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-21 09:53:12,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-21 09:53:12,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-21 09:53:12,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-21 09:53:12,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-21 09:53:12,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-21 09:53:12,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-21 09:53:12,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-21 09:53:12,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-21 09:53:12,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-21 09:53:12,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-21 09:53:12,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-21 09:53:12,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-21 09:53:12,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-21 09:53:12,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-21 09:53:12,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-21 09:53:12,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-21 09:53:12,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-21 09:53:12,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-21 09:53:12,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-21 09:53:12,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-21 09:53:12,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-21 09:53:12,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-21 09:53:12,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-21 09:53:12,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-21 09:53:12,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-21 09:53:12,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-21 09:53:12,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-21 09:53:12,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-21 09:53:12,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-21 09:53:12,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-21 09:53:12,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-21 09:53:12,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-21 09:53:12,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-21 09:53:12,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-21 09:53:12,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-21 09:53:12,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-21 09:53:12,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-21 09:53:12,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-21 09:53:12,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-21 09:53:12,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-21 09:53:12,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-21 09:53:12,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-21 09:53:12,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-21 09:53:12,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-21 09:53:12,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-21 09:53:12,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-21 09:53:12,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-21 09:53:12,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-21 09:53:12,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-21 09:53:12,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-21 09:53:12,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-21 09:53:12,421 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:12,422 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-21 09:53:12,422 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-21 09:53:12,833 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 567 ms to try 5000 different iterations.  Found a solution that moves 4685 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18006356933290285. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039206813326873244, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0592697802108307, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5205555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039206813326873244, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17566666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.238, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:12,845 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1347678568:79 , srv1173836134:80 , srv1128932182:82 , srv1217752864:82 , srv1324166898:82 , srv1362828573:82 , srv1566076759:82 , srv478383151:82 , srv771251780:82 , srv990545135:82 , srv1119901530:83 , srv1243744647:83 , srv1312770021:83 , srv1487842283:83 , srv1730179451:83 , srv650283674:83 , srv1182100125:84 , srv1548016313:84 , srv1585373032:84 , srv1866788701:84 , srv1949041295:84 , srv2006298590:84 , srv2066731760:84 , srv29595577:84 , srv359959197:84 , srv393777087:84 , srv643580617:84 , srv1157042046:85 , srv1280271603:85 , srv1353891104:85 , srv1432747485:85 , srv1739182201:85 , srv196943017:85 , srv2121748910:85 , srv26654097:85 , srv351415251:85 , srv622750390:85 , srv818626894:85 , srv883698291:85 , srv1179113431:86 , srv1269939408:86 , srv1358309262:86 , srv1406409015:86 , srv1490606202:86 , srv1571849364:86 , srv1766579067:86 , srv198846756:86 , srv2028395886:86 , srv319470219:86 , srv914272400:86 , srv1037298215:87 , srv1069741736:87 , srv1445393742:87 , srv148346263:87 , srv1704774236:87 , srv1900800830:87 , srv2124143145:87 , srv22761114:87 , srv378474511:87 , srv420522325:87 , srv585560135:87 , srv771375226:87 , srv788903913:87 , srv1028453383:88 , srv1086743529:88 , srv1093717479:88 , srv1143810146:88 , srv1211366566:88 , srv1264388015:88 , srv1300994718:88 , srv1595926573:88 , srv1678504986:88 , srv1911661909:88 , srv342223849:88 , srv440873943:88 , srv607308238:88 , srv740429402:88 , srv1152975546:89 , srv122005604:89 , srv1641011539:89 , srv1758753011:89 , srv1860730761:89 , srv2099679155:89 , srv346933771:89 , srv772430203:89 , srv1268025172:90 , srv1397558207:90 , srv1456173527:90 , srv2077435605:90 , srv454110508:90 , srv844023907:90 , srv580501338:91 , srv845204158:92 , srv1291137582:93 , srv1505584559:93 , srv1541899612:93 , srv76535791:93 , srv995309455:94 , srv19232699:95 , srv1219454509:438 }
2023-08-21 09:53:12,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-21 09:53:12,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv22761114=67, srv2121748910=65, srv342223849=71, srv1211366566=13, srv643580617=85, srv1678504986=46, srv1766579067=51, srv772430203=91, srv378474511=75, srv2124143145=66, srv1219454509=15, srv1490606202=37, srv771251780=89, srv1028453383=0, srv1739182201=49, srv1758753011=50, srv844023907=94, srv622750390=84, srv76535791=88, srv2066731760=62, srv1143810146=7, srv393777087=76, srv1173836134=10, srv1037298215=1, srv1397558207=30, srv1866788701=53, srv1243744647=17, srv1353891104=27, srv1641011539=45, srv454110508=79, srv883698291=96, srv1406409015=31, srv1280271603=21, srv148346263=35, srv995309455=99, srv1217752864=14, srv1362828573=29, srv359959197=74, srv1291137582=22, srv1445393742=33, srv990545135=98, srv1595926573=44, srv788903913=92, srv1182100125=12, srv1119901530=5, srv2006298590=60, srv1566076759=41, srv650283674=86, srv580501338=81, srv1911661909=55, srv1268025172=19, srv440873943=78, srv914272400=97, srv585560135=82, srv198846756=59, srv1432747485=32, srv122005604=16, srv1300994718=23, srv1548016313=40, srv1571849364=42, srv1324166898=25, srv1541899612=39, srv1093717479=4, srv26654097=68, srv1860730761=52, srv319470219=70, srv2028395886=61, srv1949041295=57, srv2099679155=64, srv2077435605=63, srv1505584559=38, srv771375226=90, srv1704774236=47, srv1128932182=6, srv1086743529=3, srv607308238=83, srv1487842283=36, srv29595577=69, srv1312770021=24, srv740429402=87, srv1157042046=9, srv1456173527=34, srv478383151=80, srv818626894=93, srv1179113431=11, srv1347678568=26, srv196943017=58, srv1585373032=43, srv19232699=56, srv1264388015=18, srv1269939408=20, srv1900800830=54, srv420522325=77, srv1069741736=2, srv845204158=95, srv346933771=72, srv351415251=73, srv1152975546=8, srv1358309262=28, srv1730179451=48} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-21 09:53:12,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-21 09:53:12,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-21 09:53:12,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-21 09:53:12,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-21 09:53:12,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-21 09:53:12,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-21 09:53:12,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-21 09:53:12,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-21 09:53:12,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-21 09:53:12,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-21 09:53:12,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-21 09:53:12,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-21 09:53:12,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-21 09:53:12,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-21 09:53:12,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-21 09:53:12,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-21 09:53:12,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-21 09:53:12,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-21 09:53:12,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-21 09:53:12,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-21 09:53:12,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-21 09:53:12,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-21 09:53:12,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-21 09:53:12,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-21 09:53:12,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-21 09:53:12,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-21 09:53:12,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-21 09:53:12,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-21 09:53:12,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-21 09:53:12,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-21 09:53:12,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-21 09:53:12,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-21 09:53:12,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-21 09:53:12,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-21 09:53:12,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-21 09:53:12,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-21 09:53:12,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-21 09:53:12,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-21 09:53:12,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-21 09:53:12,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-21 09:53:12,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-21 09:53:12,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-21 09:53:12,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-21 09:53:12,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-21 09:53:12,989 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039206813326873244, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0592697802108307, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039206813326873244, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17566666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.238, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:12,989 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-21 09:53:12,990 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.180030764261746, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039206813326873244, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0592697802108307, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039206813326873244, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17566666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.238, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-21 09:53:13,361 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 475 ms to try 5000 different iterations.  Found a solution that moves 2615 regions; Going from a computed imbalance of 0.180030764261746 to a new imbalance of 0.015379603907121264. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01936580216652055); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03351800695332968, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29055555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01936580216652055); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:13,375 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv995309455:76 , srv1505584559:78 , srv1541899612:78 , srv1291137582:79 , srv19232699:79 , srv76535791:79 , srv580501338:80 , srv844023907:80 , srv1456173527:81 , srv845204158:81 , srv1397558207:82 , srv1758753011:82 , srv2077435605:82 , srv454110508:82 , srv607308238:82 , srv1069741736:83 , srv1211366566:83 , srv122005604:83 , srv1264388015:83 , srv1268025172:83 , srv1445393742:83 , srv1678504986:83 , srv1704774236:83 , srv1143810146:84 , srv1595926573:84 , srv1641011539:84 , srv342223849:84 , srv346933771:84 , srv378474511:84 , srv740429402:84 , srv788903913:84 , srv1028453383:85 , srv1086743529:85 , srv1093717479:85 , srv1152975546:85 , srv1157042046:85 , srv1179113431:85 , srv1269939408:85 , srv148346263:85 , srv1860730761:85 , srv1900800830:85 , srv1911661909:85 , srv198846756:85 , srv2099679155:85 , srv1037298215:86 , srv1300994718:86 , srv1406409015:86 , srv1490606202:86 , srv1766579067:86 , srv2124143145:86 , srv319470219:86 , srv420522325:86 , srv585560135:86 , srv771375226:86 , srv818626894:86 , srv883698291:86 , srv1312770021:87 , srv1358309262:87 , srv1571849364:87 , srv2121748910:87 , srv22761114:87 , srv440873943:87 , srv772430203:87 , srv914272400:87 , srv1182100125:88 , srv1243744647:88 , srv1280271603:88 , srv1362828573:88 , srv1432747485:88 , srv1487842283:88 , srv1548016313:88 , srv1585373032:88 , srv1739182201:88 , srv1949041295:88 , srv196943017:88 , srv2006298590:88 , srv2028395886:88 , srv351415251:88 , srv359959197:88 , srv990545135:88 , srv1119901530:89 , srv1128932182:89 , srv1217752864:89 , srv1353891104:89 , srv1566076759:89 , srv29595577:89 , srv650283674:89 , srv1730179451:90 , srv26654097:90 , srv622750390:90 , srv771251780:90 , srv1173836134:91 , srv1324166898:91 , srv1866788701:91 , srv2066731760:91 , srv393777087:91 , srv478383151:91 , srv643580617:91 , srv1347678568:94 , srv1219454509:508 }
2023-08-21 09:53:13,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-21 09:53:13,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv22761114=67, srv2121748910=65, srv342223849=71, srv1211366566=13, srv643580617=85, srv1678504986=46, srv1766579067=51, srv772430203=91, srv378474511=75, srv2124143145=66, srv1219454509=15, srv1490606202=37, srv771251780=89, srv1028453383=0, srv1739182201=49, srv1758753011=50, srv844023907=94, srv622750390=84, srv76535791=88, srv2066731760=62, srv1143810146=7, srv393777087=76, srv1173836134=10, srv1037298215=1, srv1397558207=30, srv1866788701=53, srv1243744647=17, srv1353891104=27, srv1641011539=45, srv454110508=79, srv883698291=96, srv1406409015=31, srv1280271603=21, srv148346263=35, srv995309455=99, srv1217752864=14, srv1362828573=29, srv359959197=74, srv1291137582=22, srv1445393742=33, srv990545135=98, srv1595926573=44, srv788903913=92, srv1182100125=12, srv1119901530=5, srv2006298590=60, srv1566076759=41, srv650283674=86, srv580501338=81, srv1911661909=55, srv1268025172=19, srv440873943=78, srv914272400=97, srv585560135=82, srv198846756=59, srv1432747485=32, srv122005604=16, srv1300994718=23, srv1548016313=40, srv1571849364=42, srv1324166898=25, srv1541899612=39, srv1093717479=4, srv26654097=68, srv1860730761=52, srv319470219=70, srv2028395886=61, srv1949041295=57, srv2099679155=64, srv2077435605=63, srv1505584559=38, srv771375226=90, srv1704774236=47, srv1128932182=6, srv1086743529=3, srv607308238=83, srv1487842283=36, srv29595577=69, srv1312770021=24, srv740429402=87, srv1157042046=9, srv1456173527=34, srv478383151=80, srv818626894=93, srv1179113431=11, srv1347678568=26, srv196943017=58, srv1585373032=43, srv19232699=56, srv1264388015=18, srv1269939408=20, srv1900800830=54, srv420522325=77, srv1069741736=2, srv845204158=95, srv346933771=72, srv351415251=73, srv1152975546=8, srv1358309262=28, srv1730179451=48} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-21 09:53:13,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-21 09:53:13,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-21 09:53:13,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-21 09:53:13,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-21 09:53:13,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-21 09:53:13,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-21 09:53:13,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-21 09:53:13,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-21 09:53:13,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-21 09:53:13,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-21 09:53:13,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-21 09:53:13,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-21 09:53:13,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-21 09:53:13,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-21 09:53:13,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-21 09:53:13,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-21 09:53:13,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-21 09:53:13,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-21 09:53:13,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-21 09:53:13,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-21 09:53:13,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-21 09:53:13,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-21 09:53:13,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-21 09:53:13,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-21 09:53:13,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-21 09:53:13,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-21 09:53:13,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-21 09:53:13,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-21 09:53:13,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-21 09:53:13,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-21 09:53:13,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-21 09:53:13,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-21 09:53:13,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-21 09:53:13,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-21 09:53:13,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-21 09:53:13,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-21 09:53:13,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-21 09:53:13,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-21 09:53:13,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-21 09:53:13,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-21 09:53:13,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-21 09:53:13,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-21 09:53:13,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-21 09:53:13,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-21 09:53:13,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-21 09:53:13,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-21 09:53:13,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-21 09:53:13,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-21 09:53:13,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-21 09:53:13,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-21 09:53:13,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-21 09:53:13,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-21 09:53:13,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-21 09:53:13,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-21 09:53:13,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-21 09:53:13,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-21 09:53:13,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-21 09:53:13,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-21 09:53:13,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-21 09:53:13,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-21 09:53:13,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-21 09:53:13,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-21 09:53:13,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-21 09:53:13,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-21 09:53:13,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-21 09:53:13,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-21 09:53:13,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-21 09:53:13,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-21 09:53:13,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-21 09:53:13,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-21 09:53:13,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-21 09:53:13,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-21 09:53:13,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-21 09:53:13,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-21 09:53:13,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-21 09:53:13,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-21 09:53:13,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-21 09:53:13,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-21 09:53:13,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-21 09:53:13,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-21 09:53:13,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-21 09:53:13,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-21 09:53:13,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-21 09:53:13,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-21 09:53:13,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-21 09:53:13,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-21 09:53:13,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-21 09:53:13,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-21 09:53:13,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-21 09:53:13,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-21 09:53:13,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-21 09:53:13,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-21 09:53:13,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-21 09:53:13,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-21 09:53:13,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-21 09:53:13,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-21 09:53:13,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-21 09:53:13,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-21 09:53:13,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-21 09:53:13,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-21 09:53:13,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-21 09:53:13,585 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01936580216652055); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03351800695332968, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01936580216652055); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:13,585 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-21 09:53:13,585 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015361293285760505, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01936580216652055); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03351800695332968, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01936580216652055); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-21 09:53:13,916 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 473 ms to try 5000 different iterations.  Found a solution that moves 1038 regions; Going from a computed imbalance of 0.015361293285760505 to a new imbalance of 7.44577542292288E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0047535601087442675); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00704323668071124); MoveCostFunction : (multiplier=7.0, imbalance=0.11533333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0047535601087442675); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00425); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:13,918 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv643580617:82 , srv772430203:82 , srv1300994718:83 , srv1866788701:83 , srv2066731760:83 , srv22761114:83 , srv26654097:83 , srv393777087:83 , srv440873943:83 , srv622750390:83 , srv1093717479:84 , srv1152975546:84 , srv1324166898:84 , srv1347678568:84 , srv1353891104:84 , srv1432747485:84 , srv1571849364:84 , srv1730179451:84 , srv1860730761:84 , srv19232699:84 , srv2028395886:84 , srv2099679155:84 , srv29595577:84 , srv420522325:84 , srv585560135:84 , srv771375226:84 , srv845204158:84 , srv914272400:84 , srv1028453383:85 , srv1037298215:85 , srv1086743529:85 , srv1143810146:85 , srv122005604:85 , srv1268025172:85 , srv1280271603:85 , srv1358309262:85 , srv1397558207:85 , srv1406409015:85 , srv1548016313:85 , srv1585373032:85 , srv1595926573:85 , srv1641011539:85 , srv1739182201:85 , srv1900800830:85 , srv1911661909:85 , srv196943017:85 , srv2006298590:85 , srv2077435605:85 , srv2121748910:85 , srv2124143145:85 , srv346933771:85 , srv351415251:85 , srv454110508:85 , srv478383151:85 , srv650283674:85 , srv1119901530:86 , srv1173836134:86 , srv1182100125:86 , srv1217752864:86 , srv1243744647:86 , srv1269939408:86 , srv1291137582:86 , srv1456173527:86 , srv1487842283:86 , srv1490606202:86 , srv1541899612:86 , srv1758753011:86 , srv1766579067:86 , srv1949041295:86 , srv198846756:86 , srv319470219:86 , srv342223849:86 , srv359959197:86 , srv740429402:86 , srv76535791:86 , srv771251780:86 , srv1069741736:87 , srv1157042046:87 , srv1179113431:87 , srv1211366566:87 , srv1264388015:87 , srv1312770021:87 , srv1362828573:87 , srv1445393742:87 , srv148346263:87 , srv1505584559:87 , srv1566076759:87 , srv1678504986:87 , srv1704774236:87 , srv378474511:87 , srv580501338:87 , srv607308238:87 , srv788903913:87 , srv844023907:87 , srv883698291:87 , srv1128932182:88 , srv818626894:88 , srv990545135:88 , srv995309455:88 , srv1219454509:554 }
2023-08-21 09:53:13,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-21 09:53:13,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv22761114=67, srv2121748910=65, srv342223849=71, srv1211366566=13, srv643580617=85, srv1678504986=46, srv1766579067=51, srv772430203=91, srv378474511=75, srv2124143145=66, srv1219454509=15, srv1490606202=37, srv771251780=89, srv1028453383=0, srv1739182201=49, srv1758753011=50, srv844023907=94, srv622750390=84, srv76535791=88, srv2066731760=62, srv1143810146=7, srv393777087=76, srv1173836134=10, srv1037298215=1, srv1397558207=30, srv1866788701=53, srv1243744647=17, srv1353891104=27, srv1641011539=45, srv454110508=79, srv883698291=96, srv1406409015=31, srv1280271603=21, srv148346263=35, srv995309455=99, srv1217752864=14, srv1362828573=29, srv359959197=74, srv1291137582=22, srv1445393742=33, srv990545135=98, srv1595926573=44, srv788903913=92, srv1182100125=12, srv1119901530=5, srv2006298590=60, srv1566076759=41, srv650283674=86, srv580501338=81, srv1911661909=55, srv1268025172=19, srv440873943=78, srv914272400=97, srv585560135=82, srv198846756=59, srv1432747485=32, srv122005604=16, srv1300994718=23, srv1548016313=40, srv1571849364=42, srv1324166898=25, srv1541899612=39, srv1093717479=4, srv26654097=68, srv1860730761=52, srv319470219=70, srv2028395886=61, srv1949041295=57, srv2099679155=64, srv2077435605=63, srv1505584559=38, srv771375226=90, srv1704774236=47, srv1128932182=6, srv1086743529=3, srv607308238=83, srv1487842283=36, srv29595577=69, srv1312770021=24, srv740429402=87, srv1157042046=9, srv1456173527=34, srv478383151=80, srv818626894=93, srv1179113431=11, srv1347678568=26, srv196943017=58, srv1585373032=43, srv19232699=56, srv1264388015=18, srv1269939408=20, srv1900800830=54, srv420522325=77, srv1069741736=2, srv845204158=95, srv346933771=72, srv351415251=73, srv1152975546=8, srv1358309262=28, srv1730179451=48} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-21 09:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-21 09:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-21 09:53:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-21 09:53:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-21 09:53:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-21 09:53:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-21 09:53:13,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-21 09:53:13,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-21 09:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-21 09:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-21 09:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-21 09:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-21 09:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-21 09:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-21 09:53:13,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-21 09:53:13,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-21 09:53:13,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-21 09:53:13,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-21 09:53:13,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-21 09:53:13,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-21 09:53:13,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-21 09:53:13,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-21 09:53:13,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-21 09:53:13,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-21 09:53:13,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-21 09:53:13,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-21 09:53:13,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-21 09:53:13,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-21 09:53:13,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-21 09:53:14,062 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0047535601087442675); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00704323668071124); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0047535601087442675); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00425); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:14,062 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-21 09:53:14,062 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.373093109452644E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0047535601087442675); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00704323668071124); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0047535601087442675); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00425); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-21 09:53:14,454 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 503 ms to try 5000 different iterations.  Found a solution that moves 237 regions; Going from a computed imbalance of 7.373093109452644E-4 to a new imbalance of 3.5227026367944564E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.026333333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:14,455 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1128932182:83 , srv148346263:83 , srv342223849:83 , srv772430203:83 , srv818626894:83 , srv995309455:83 , srv1037298215:84 , srv1086743529:84 , srv1119901530:84 , srv1143810146:84 , srv1152975546:84 , srv1179113431:84 , srv1182100125:84 , srv1211366566:84 , srv1264388015:84 , srv1268025172:84 , srv1280271603:84 , srv1291137582:84 , srv1353891104:84 , srv1358309262:84 , srv1490606202:84 , srv1505584559:84 , srv1566076759:84 , srv1641011539:84 , srv1678504986:84 , srv1739182201:84 , srv1766579067:84 , srv1866788701:84 , srv1911661909:84 , srv19232699:84 , srv1949041295:84 , srv196943017:84 , srv198846756:84 , srv2028395886:84 , srv2066731760:84 , srv2099679155:84 , srv2121748910:84 , srv26654097:84 , srv319470219:84 , srv346933771:84 , srv351415251:84 , srv359959197:84 , srv393777087:84 , srv454110508:84 , srv478383151:84 , srv580501338:84 , srv622750390:84 , srv740429402:84 , srv771251780:84 , srv788903913:84 , srv883698291:84 , srv990545135:84 , srv1028453383:85 , srv1069741736:85 , srv1093717479:85 , srv1157042046:85 , srv1217752864:85 , srv122005604:85 , srv1243744647:85 , srv1269939408:85 , srv1300994718:85 , srv1312770021:85 , srv1324166898:85 , srv1347678568:85 , srv1362828573:85 , srv1397558207:85 , srv1406409015:85 , srv1432747485:85 , srv1445393742:85 , srv1456173527:85 , srv1487842283:85 , srv1541899612:85 , srv1548016313:85 , srv1571849364:85 , srv1585373032:85 , srv1595926573:85 , srv1704774236:85 , srv1730179451:85 , srv1758753011:85 , srv1860730761:85 , srv1900800830:85 , srv2006298590:85 , srv2077435605:85 , srv2124143145:85 , srv22761114:85 , srv29595577:85 , srv378474511:85 , srv420522325:85 , srv440873943:85 , srv585560135:85 , srv607308238:85 , srv643580617:85 , srv650283674:85 , srv76535791:85 , srv771375226:85 , srv844023907:85 , srv845204158:85 , srv914272400:85 , srv1173836134:86 , srv1219454509:642 }
2023-08-21 09:53:14,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-21 09:53:14,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv22761114=67, srv2121748910=65, srv342223849=71, srv1211366566=13, srv643580617=85, srv1678504986=46, srv1766579067=51, srv772430203=91, srv378474511=75, srv2124143145=66, srv1219454509=15, srv1490606202=37, srv771251780=89, srv1028453383=0, srv1739182201=49, srv1758753011=50, srv844023907=94, srv622750390=84, srv76535791=88, srv2066731760=62, srv1143810146=7, srv393777087=76, srv1173836134=10, srv1037298215=1, srv1397558207=30, srv1866788701=53, srv1243744647=17, srv1353891104=27, srv1641011539=45, srv454110508=79, srv883698291=96, srv1406409015=31, srv1280271603=21, srv148346263=35, srv995309455=99, srv1217752864=14, srv1362828573=29, srv359959197=74, srv1291137582=22, srv1445393742=33, srv990545135=98, srv1595926573=44, srv788903913=92, srv1182100125=12, srv1119901530=5, srv2006298590=60, srv1566076759=41, srv650283674=86, srv580501338=81, srv1911661909=55, srv1268025172=19, srv440873943=78, srv914272400=97, srv585560135=82, srv198846756=59, srv1432747485=32, srv122005604=16, srv1300994718=23, srv1548016313=40, srv1571849364=42, srv1324166898=25, srv1541899612=39, srv1093717479=4, srv26654097=68, srv1860730761=52, srv319470219=70, srv2028395886=61, srv1949041295=57, srv2099679155=64, srv2077435605=63, srv1505584559=38, srv771375226=90, srv1704774236=47, srv1128932182=6, srv1086743529=3, srv607308238=83, srv1487842283=36, srv29595577=69, srv1312770021=24, srv740429402=87, srv1157042046=9, srv1456173527=34, srv478383151=80, srv818626894=93, srv1179113431=11, srv1347678568=26, srv196943017=58, srv1585373032=43, srv19232699=56, srv1264388015=18, srv1269939408=20, srv1900800830=54, srv420522325=77, srv1069741736=2, srv845204158=95, srv346933771=72, srv351415251=73, srv1152975546=8, srv1358309262=28, srv1730179451=48} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-21 09:53:14,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-21 09:53:14,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-21 09:53:14,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-21 09:53:14,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-21 09:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-21 09:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-21 09:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-21 09:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-21 09:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-21 09:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-21 09:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-21 09:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-21 09:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-21 09:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-21 09:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-21 09:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-21 09:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-21 09:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-21 09:53:14,605 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:14,606 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-21 09:53:14,606 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.356751689853746E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-21 09:53:14,863 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 375 ms to try 5000 different iterations.  Found a solution that moves 25 regions; Going from a computed imbalance of 3.356751689853746E-5 to a new imbalance of 7.5628112418111995E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.002777777777777778); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:14,864 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1028453383:83 , srv1173836134:83 , srv1860730761:83 , srv2124143145:83 , srv378474511:83 , srv440873943:83 , srv76535791:83 , srv1037298215:84 , srv1069741736:84 , srv1086743529:84 , srv1093717479:84 , srv1119901530:84 , srv1128932182:84 , srv1152975546:84 , srv1157042046:84 , srv1179113431:84 , srv1182100125:84 , srv1211366566:84 , srv1217752864:84 , srv122005604:84 , srv1243744647:84 , srv1264388015:84 , srv1268025172:84 , srv1269939408:84 , srv1280271603:84 , srv1291137582:84 , srv1300994718:84 , srv1312770021:84 , srv1324166898:84 , srv1347678568:84 , srv1353891104:84 , srv1358309262:84 , srv1362828573:84 , srv1397558207:84 , srv1406409015:84 , srv1432747485:84 , srv1445393742:84 , srv1456173527:84 , srv148346263:84 , srv1487842283:84 , srv1490606202:84 , srv1505584559:84 , srv1541899612:84 , srv1548016313:84 , srv1566076759:84 , srv1571849364:84 , srv1585373032:84 , srv1595926573:84 , srv1641011539:84 , srv1678504986:84 , srv1704774236:84 , srv1730179451:84 , srv1739182201:84 , srv1758753011:84 , srv1766579067:84 , srv1866788701:84 , srv1900800830:84 , srv1911661909:84 , srv19232699:84 , srv1949041295:84 , srv196943017:84 , srv2006298590:84 , srv2028395886:84 , srv2066731760:84 , srv2077435605:84 , srv2099679155:84 , srv22761114:84 , srv26654097:84 , srv29595577:84 , srv319470219:84 , srv346933771:84 , srv351415251:84 , srv359959197:84 , srv393777087:84 , srv420522325:84 , srv478383151:84 , srv580501338:84 , srv585560135:84 , srv607308238:84 , srv622750390:84 , srv643580617:84 , srv650283674:84 , srv740429402:84 , srv771251780:84 , srv771375226:84 , srv788903913:84 , srv818626894:84 , srv844023907:84 , srv845204158:84 , srv883698291:84 , srv914272400:84 , srv990545135:84 , srv1143810146:85 , srv198846756:85 , srv2121748910:85 , srv342223849:85 , srv454110508:85 , srv772430203:85 , srv995309455:85 , srv1219454509:684 }
2023-08-21 09:53:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-21 09:53:14,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv22761114=67, srv2121748910=65, srv342223849=71, srv1211366566=13, srv643580617=85, srv1678504986=46, srv1766579067=51, srv772430203=91, srv378474511=75, srv2124143145=66, srv1219454509=15, srv1490606202=37, srv771251780=89, srv1028453383=0, srv1739182201=49, srv1758753011=50, srv844023907=94, srv622750390=84, srv76535791=88, srv2066731760=62, srv1143810146=7, srv393777087=76, srv1173836134=10, srv1037298215=1, srv1397558207=30, srv1866788701=53, srv1243744647=17, srv1353891104=27, srv1641011539=45, srv454110508=79, srv883698291=96, srv1406409015=31, srv1280271603=21, srv148346263=35, srv995309455=99, srv1217752864=14, srv1362828573=29, srv359959197=74, srv1291137582=22, srv1445393742=33, srv990545135=98, srv1595926573=44, srv788903913=92, srv1182100125=12, srv1119901530=5, srv2006298590=60, srv1566076759=41, srv650283674=86, srv580501338=81, srv1911661909=55, srv1268025172=19, srv440873943=78, srv914272400=97, srv585560135=82, srv198846756=59, srv1432747485=32, srv122005604=16, srv1300994718=23, srv1548016313=40, srv1571849364=42, srv1324166898=25, srv1541899612=39, srv1093717479=4, srv26654097=68, srv1860730761=52, srv319470219=70, srv2028395886=61, srv1949041295=57, srv2099679155=64, srv2077435605=63, srv1505584559=38, srv771375226=90, srv1704774236=47, srv1128932182=6, srv1086743529=3, srv607308238=83, srv1487842283=36, srv29595577=69, srv1312770021=24, srv740429402=87, srv1157042046=9, srv1456173527=34, srv478383151=80, srv818626894=93, srv1179113431=11, srv1347678568=26, srv196943017=58, srv1585373032=43, srv19232699=56, srv1264388015=18, srv1269939408=20, srv1900800830=54, srv420522325=77, srv1069741736=2, srv845204158=95, srv346933771=72, srv351415251=73, srv1152975546=8, srv1358309262=28, srv1730179451=48} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-21 09:53:14,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-21 09:53:14,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-21 09:53:14,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-21 09:53:14,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-21 09:53:14,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-21 09:53:14,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-21 09:53:14,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-21 09:53:14,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-21 09:53:14,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-21 09:53:14,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-21 09:53:14,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-21 09:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-21 09:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-21 09:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-21 09:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-21 09:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-21 09:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-21 09:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-21 09:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-21 09:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-21 09:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-21 09:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-21 09:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-21 09:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-21 09:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-21 09:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-21 09:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-21 09:53:15,007 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:15,008 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1028453383:83 , srv1173836134:83 , srv1860730761:83 , srv2124143145:83 , srv378474511:83 , srv440873943:83 , srv76535791:83 , srv1037298215:84 , srv1069741736:84 , srv1086743529:84 , srv1093717479:84 , srv1119901530:84 , srv1128932182:84 , srv1152975546:84 , srv1157042046:84 , srv1179113431:84 , srv1182100125:84 , srv1211366566:84 , srv1217752864:84 , srv122005604:84 , srv1243744647:84 , srv1264388015:84 , srv1268025172:84 , srv1269939408:84 , srv1280271603:84 , srv1291137582:84 , srv1300994718:84 , srv1312770021:84 , srv1324166898:84 , srv1347678568:84 , srv1353891104:84 , srv1358309262:84 , srv1362828573:84 , srv1397558207:84 , srv1406409015:84 , srv1432747485:84 , srv1445393742:84 , srv1456173527:84 , srv148346263:84 , srv1487842283:84 , srv1490606202:84 , srv1505584559:84 , srv1541899612:84 , srv1548016313:84 , srv1566076759:84 , srv1571849364:84 , srv1585373032:84 , srv1595926573:84 , srv1641011539:84 , srv1678504986:84 , srv1704774236:84 , srv1730179451:84 , srv1739182201:84 , srv1758753011:84 , srv1766579067:84 , srv1866788701:84 , srv1900800830:84 , srv1911661909:84 , srv19232699:84 , srv1949041295:84 , srv196943017:84 , srv2006298590:84 , srv2028395886:84 , srv2066731760:84 , srv2077435605:84 , srv2099679155:84 , srv22761114:84 , srv26654097:84 , srv29595577:84 , srv319470219:84 , srv346933771:84 , srv351415251:84 , srv359959197:84 , srv393777087:84 , srv420522325:84 , srv478383151:84 , srv580501338:84 , srv585560135:84 , srv607308238:84 , srv622750390:84 , srv643580617:84 , srv650283674:84 , srv740429402:84 , srv771251780:84 , srv771375226:84 , srv788903913:84 , srv818626894:84 , srv844023907:84 , srv845204158:84 , srv883698291:84 , srv914272400:84 , srv990545135:84 , srv1143810146:85 , srv198846756:85 , srv2121748910:85 , srv342223849:85 , srv454110508:85 , srv772430203:85 , srv995309455:85 , srv1219454509:684 }
2023-08-21 09:53:15,041 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=359 (was 338) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 172), AvailableMemoryMB=10520 (was 11354)
2023-08-21 09:53:15,049 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=359, ProcessCount=172, AvailableMemoryMB=10519
2023-08-21 09:53:15,050 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-21 09:53:15,051 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-21 09:53:15,066 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100726465:84 , srv1016320341:84 , srv1126527381:84 , srv1153196382:84 , srv1155498705:84 , srv1208628960:84 , srv1260932349:84 , srv1317934580:84 , srv1353026711:84 , srv1390344529:84 , srv1597239643:84 , srv1674673966:84 , srv1790863372:84 , srv1827674795:84 , srv1845114063:84 , srv1902135385:84 , srv1978493238:84 , srv2010893314:84 , srv2087333935:84 , srv2104082063:84 , srv2109256630:84 , srv2136574713:84 , srv32874638:84 , srv382704946:84 , srv412807124:84 , srv454215046:84 , srv470259378:84 , srv701584283:84 , srv804728979:84 , srv423992070:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-21 09:53:15,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-21 09:53:15,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1845114063=14, srv100726465=0, srv32874638=22, srv1390344529=9, srv2010893314=17, srv382704946=23, srv804728979=29, srv1153196382=3, srv1155498705=4, srv1353026711=8, srv1260932349=6, srv412807124=24, srv1016320341=1, srv2104082063=19, srv1317934580=7, srv1827674795=13, srv701584283=28, srv1597239643=10, srv423992070=25, srv1674673966=11, srv2087333935=18, srv1978493238=16, srv1126527381=2, srv2136574713=21, srv1208628960=5, srv2109256630=20, srv1902135385=15, srv454215046=26, srv470259378=27, srv1790863372=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-21 09:53:15,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-21 09:53:15,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-21 09:53:15,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-21 09:53:15,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-21 09:53:15,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-21 09:53:15,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-21 09:53:15,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-21 09:53:15,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-21 09:53:15,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-21 09:53:15,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-21 09:53:15,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-21 09:53:15,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-21 09:53:15,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-21 09:53:15,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-21 09:53:15,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-21 09:53:15,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-21 09:53:15,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-21 09:53:15,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-21 09:53:15,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-21 09:53:15,103 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1120749707284447, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:53:15,103 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912546187837461, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1120749707284447, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-21 09:55:15,090 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6130430 different iterations.  Found a solution that moves 1844 regions; Going from a computed imbalance of 0.9912546187837461 to a new imbalance of 4.303987990979898E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.682962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:55:15,097 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv100726465:90 , srv1016320341:90 , srv1126527381:90 , srv1153196382:90 , srv1155498705:90 , srv1208628960:90 , srv1260932349:90 , srv1317934580:90 , srv1353026711:90 , srv1390344529:90 , srv1597239643:90 , srv1674673966:90 , srv1790863372:90 , srv1827674795:90 , srv1845114063:90 , srv1902135385:90 , srv1978493238:90 , srv2010893314:90 , srv2087333935:90 , srv2104082063:90 , srv2109256630:90 , srv2136574713:90 , srv32874638:90 , srv382704946:90 , srv412807124:90 , srv423992070:90 , srv454215046:90 , srv470259378:90 , srv701584283:90 , srv804728979:90 }
2023-08-21 09:55:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-21 09:55:15,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1845114063=14, srv100726465=0, srv32874638=22, srv1390344529=9, srv2010893314=17, srv382704946=23, srv804728979=29, srv1153196382=3, srv1155498705=4, srv1353026711=8, srv1260932349=6, srv412807124=24, srv1016320341=1, srv2104082063=19, srv1317934580=7, srv1827674795=13, srv701584283=28, srv1597239643=10, srv423992070=25, srv1674673966=11, srv2087333935=18, srv1978493238=16, srv1126527381=2, srv2136574713=21, srv1208628960=5, srv2109256630=20, srv1902135385=15, srv454215046=26, srv470259378=27, srv1790863372=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-21 09:55:15,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-21 09:55:15,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-21 09:55:15,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-21 09:55:15,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-21 09:55:15,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-21 09:55:15,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-21 09:55:15,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-21 09:55:15,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-21 09:55:15,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-21 09:55:15,122 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-21 09:55:15,123 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv100726465:90 , srv1016320341:90 , srv1126527381:90 , srv1153196382:90 , srv1155498705:90 , srv1208628960:90 , srv1260932349:90 , srv1317934580:90 , srv1353026711:90 , srv1390344529:90 , srv1597239643:90 , srv1674673966:90 , srv1790863372:90 , srv1827674795:90 , srv1845114063:90 , srv1902135385:90 , srv1978493238:90 , srv2010893314:90 , srv2087333935:90 , srv2104082063:90 , srv2109256630:90 , srv2136574713:90 , srv32874638:90 , srv382704946:90 , srv412807124:90 , srv423992070:90 , srv454215046:90 , srv470259378:90 , srv701584283:90 , srv804728979:90 }
2023-08-21 09:55:15,140 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=206 (was 359), ProcessCount=178 (was 172) - ProcessCount LEAK? -, AvailableMemoryMB=10799 (was 10519) - AvailableMemoryMB LEAK? -